What impacts what we remember about objects we have just encountered? Influential theories of learning suggest that more active engagement leads to stronger memories than passive observation. However, it is not clear which aspects of interaction lead to stronger memories, nor what kinds of memories are supported by active engagement. Here we conduct several experiments to investigate the impact of assembling an object on subsequent recognition and recall performance. We found that reconstructing a block tower by copying it part-by-part could impair subsequent memory for that tower, compared to passively viewing that tower. By contrast, when participants initially encoded each tower by building it from working memory, their subsequent recall was enhanced relative to when they held the tower in working memory without building it. Together our results suggest a complex relationship between the nature of our interactions with objects and our subsequent memories of them.

Compositionality is a core feature of human cognition and behavior. People readily decompose visual objects into parts and complex procedures into subtasks. Here we investigate how these two abilities interact to support learning in a block-tower assembly experiment. We measured the way participants segmented these towers based on shape information alone, and asked how well the resulting parts explained the procedures other participants used to build them. We found that people decomposed these shapes in consistent ways and the most common parts appeared especially frequently as subroutines in the assembly experiment. Moreover, we found that the subroutines participants used converged over time, reflecting shared biases toward certain ways of reconstructing each tower. More broadly, our findings suggest important similarities between the perceptual and procedural abstractions humans use to perceive and interact with objects in their environment.

Many real-world tasks require agents to coordinate their behavior to achieve shared goals. Successful collaboration requires not only adopting the same communicative conventions, but also grounding these conventions in the same task-appropriate conceptual abstractions. We investigate how humans use natural language to collaboratively solve physical assembly problems more effectively over time. Human participants were paired up in an online environment to reconstruct scenes containing two block towers. One participant could see the target towers, and sent assembly instructions for the other participant to reconstruct. Participants provided increasingly concise instructions across repeated attempts on each pair of towers, using more abstract referring expressions that captured each scene's hierarchical structure. To explain these findings, we extend recent probabilistic models of ad hoc convention formation with an explicit perceptual learning mechanism. These results shed light on the inductive biases that enable intelligent agents to coordinate upon shared procedural abstractions.

Our understanding of the visual world goes beyond naming objects, encompassing our ability to parse objects into meaningful parts, attributes, and relations. In this work, we leverage natural language descriptions for a diverse set of 2K procedurally generated objects to identify the parts people use and the principles leading these parts to be favored over others.We formalize our problem as search over a space of program libraries that contain different part concepts, using tools from machine translation to evaluate how well programs expressed in each library align to human language. By combining naturalistic language at scale with structured program representations, we discover a fundamental information-theoretic tradeoff governing the part concepts people name: people favor a lexicon that allows concise descriptions of each object, while also minimizing the size of the lexicon itself.