UC Davis

Multisensory experiences are ubiquitous in our everyday lives and impact what sensory information we notice, pay attention to, and remember. However, many areas of cognitive psychology focus on the senses individually, and/or use simplistic versions of real objects. This makes it difficult to understand whether, and how, laboratory findings can explain cognitive processing in the real world. This dissertation investigates how naturalistic object properties, including multiple sensory modalities, semantic information, and dynamic motion, contribute to sensory processing and memory formation. Chapter 2 examines the impact of task-irrelevant, semantically congruent sounds on visual recognition memory. Through a series of experiments, it demonstrates that congruent object-sound pairings facilitate recollection-based recognition and promote the formation of multisensory memories. These findings underscore the importance of considering multisensory interactions in developing models of memory applicable to real-world settings. Building on these insights, Chapter 3 investigates how multisensory object processing affects memory for nearby visual objects and scene contexts. While the presence of audiovisual objects at encoding did not significantly benefit memory for nearby visual objects, it did improve recall of the environmental context. These results highlight the broader influence of multisensory processing on episodic memory formation beyond individual objects. Chapter 4 explores the audiovisual ventriloquist effect using realistic stimuli in virtual reality. In this study, we found that animated, semantically congruent audiovisual stimuli show enhanced spatial ventriloquism at small disparities relative to the simplistic stimuli frequently used in laboratory studies of multisensory integration. The study emphasizes the role of stimulus realism and dynamic motion in audiovisual integration. Collectively, this research advances our understanding of how multisensory experiences shape memory and perception in naturalistic settings.