Frequent words tend to be short, and many researchers haveproposed that this relationship reflects a tendency towards ef-ficient communication. Recent work has sought to formalizethis observation in the context of information theory, which es-tablishes a limit on communicative efficiency called the chan-nel capacity. In this paper, I first show that the compositionalstructure of natural language prevents natural language com-munication from getting close to the channel capacity, but thata different limit, which incorporates probability in context,may be achievable. Next, I present two corpus studies in threetypologically-diverse languages that provide evidence that lan-guages change over time towards the achievable limit. Theseresults suggest that natural language optimizes for efficiencyover time, and does so in a way that is appopriate for compo-sitional codes.

Child-directed speech (CDS) is a talking style adopted by caregivers when they talk to toddlers (Snow, 1995). We consider the role of distributional semantic features of CDS in language acquisition. We view semantic structure as a manifold on which words lie. We compare the semantic structure of verbs in CDS to the semantic structure of child speech (CS) and adult-directed speech (ADS) by measuring how easy it is to align the manifolds. We find that it is easier to align verbs in CS to CDS than to align CS to ADS, suggesting that the semantic structure of CDS is reflected in child productions. We also find, by measuring verbs vertex degrees in a semantic graph, that a mixed initialized set of verbs with high degrees and medium degrees has the best performance among all alignments, suggesting that both semantic generality and diversity may be important for developing semantic representations.