Search

Thesis
Peer Reviewed

Differences in Neural Mechanisms for Auditory and Visual Rhythm Processing

Comstock, Daniel
Advisor(s): Balasubramaniam, Ramesh

UC Merced Electronic Theses and Dissertations (2021)

Only humans and a few other species of animals share the ability to reliably entrain and synchronize to rhythms. Accurately synchronizing to rhythms, as in tapping a foot to a metronome, requires the ability to precisely predict the onset of each rhythmic event. Curiously, humans have much greater difficulty synchronizing to flashing visual rhythms compared to auditory rhythms, even when both stimuli contain exactly the same timing information. While it is known that the auditory system works in conjunction with the motor system for the timing processing needed for rhythm perception, it has not been clear how the visual system processes rhythm timing. In collaboration with my co- authors, I present evidence to explain this discrepancy by showing the visual system has its own internal timing capabilities, which are separate than those utilized in auditory rhythms. I show that the visual system is able to detect perturbations of visual rhythms, but that error detection is not able to be translated into effective error correction in a synchronization task as is the case with auditory rhythms (Chapter 2). From an experiment in passive rhythm perception, I show evidence of visual timing activity arising from the visual system, as well as evidence of separate timing networks for auditory and visual rhythm processing (Chapter 3). I also show evidence that visual system timing information, in the form of neural entrainment, is present in the motor system at similar levels as auditory neural entrainment. Although visual timing information is present in the motor system, it is not able to be utilized in the same capacity as auditory rhythm timing in the motor system (Chapter 4). Taken together, these results suggest the visual system is processing rhythm timing through a visual modality specific mechanism. This mechanism is not as capable for rhythm processing as the motor system utilized in auditory rhythm processing, which results in poorer visual rhythm timing capabilities for visual rhythms compared to auditory rhythms.This dissertation, Differences in Neural Mechanisms for Auditory and Visual Rhythm Processing, is submitted by Daniel C. Comstock in the summer of 2021 in partial fulfillment of the degree Doctor of Philosophy in Cognitive and Information Sciences at the University of California, Merced, under the guidance of Ramesh Balasubramaniam.

Cover page: Differences in Neural Mechanisms for Auditory and Visual Rhythm Processing

Creative Commons 'BY' version 4.0 license

Article
Peer Reviewed

Sensorimotor Synchronization With Auditory and Visual Modalities: Behavioral and Neural Differences.

UC Merced Previously Published Works (2018)

It has long been known that the auditory system is better suited to guide temporally precise behaviors like sensorimotor synchronization (SMS) than the visual system. Although this phenomenon has been studied for many years, the underlying neural and computational mechanisms remain unclear. Growing consensus suggests the existence of multiple, interacting, context-dependent systems, and that reduced precision in visuo-motor timing might be due to the way experimental tasks have been conceived. Indeed, the appropriateness of the stimulus for a given task greatly influences timing performance. In this review, we examine timing differences for sensorimotor synchronization and error correction with auditory and visual sequences, to inspect the underlying neural mechanisms that contribute to modality differences in timing. The disparity between auditory and visual timing likely relates to differences in the processing specialization between auditory and visual modalities (temporal vs. spatial). We propose this difference could offer potential explanation for the differing temporal abilities between modalities. We also offer suggestions as to how these sensory systems interface with motor and timing systems.

Cover page: Sensorimotor Synchronization With Auditory and Visual Modalities: Behavioral and Neural Differences.

Article
Peer Reviewed

Tracking differential activation of primary and supplementary motor cortex across timing tasks: An fNIRS validation study

UC Merced Previously Published Works (2020)

Functional near-infrared spectroscopy (fNIRS) provides an alternative to functional magnetic resonance imaging (fMRI) for assessing changes in cortical hemodynamics. To establish the utility of fNIRS for measuring differential recruitment of the motor network during the production of timing-based actions, we measured cortical hemodynamic responses in 10 healthy adults while they performed two versions of a finger-tapping task. The task, used in an earlier fMRI study (Jantzen et al., 2004), was designed to track the neural basis of different timing behaviors. Participants paced their tapping to a metronomic tone, then continued tapping at the established pace without the tone. Initial tapping was either synchronous or syncopated relative to the tone. This produced a 2 × 2 design: synchronous or syncopated tapping and pacing the tapping with or continuing without a tone. Accuracy of the timing of tapping was tracked while cortical hemodynamics were monitored using fNIRS. Hemodynamic responses were computed by canonical statistical analysis across trials in each of the four conditions. Task-induced brain activation resulted in significant increases in oxygenated hemoglobin concentration (oxy-Hb) in a broad region in and around the motor cortex. Overall, syncopated tapping was harder behaviorally and produced more cortical activation than synchronous tapping. Thus, we observed significant changes in oxy-Hb in direct relation to the complexity of the task.

Article
Peer Reviewed

Tracking differential activation of primary and supplementary motor cortex across timing tasks: An fNIRS validation study

UC Merced Previously Published Works (2020)

Functional near-infrared spectroscopy (fNIRS) provides an alternative to functional magnetic resonance imaging (fMRI) for assessing changes in cortical hemodynamics. To establish the utility of fNIRS for measuring differential recruitment of the motor network during the production of timing-based actions, we measured cortical hemodynamic responses in 10 healthy adults while they performed two versions of a finger-tapping task. The task, used in an earlier fMRI study (Jantzen et al., 2004), was designed to track the neural basis of different timing behaviors. Participants paced their tapping to a metronomic tone, then continued tapping at the established pace without the tone. Initial tapping was either synchronous or syncopated relative to the tone. This produced a 2 × 2 design: synchronous or syncopated tapping and pacing the tapping with or continuing without a tone. Accuracy of the timing of tapping was tracked while cortical hemodynamics were monitored using fNIRS. Hemodynamic responses were computed by canonical statistical analysis across trials in each of the four conditions. Task-induced brain activation resulted in significant increases in oxygenated hemoglobin concentration (oxy-Hb) in a broad region in and around the motor cortex. Overall, syncopated tapping was harder behaviorally and produced more cortical activation than synchronous tapping. Thus, we observed significant changes in oxy-Hb in direct relation to the complexity of the task.

Article
Peer Reviewed

Motor and Predictive Processes in Auditory Beat and Rhythm Perception

UC Merced Previously Published Works (2020)

In this article, we review recent advances in research on rhythm and musical beat perception, focusing on the role of predictive processes in auditory motor interactions. We suggest that experimental evidence of the motor system's role in beat perception, including in passive listening, may be explained by the generation and maintenance of internal predictive models, concordant with the Active Inference framework of sensory processing. We highlight two complementary hypotheses for the neural underpinnings of rhythm perception: The Action Simulation for Auditory Prediction hypothesis (Patel and Iversen, 2014) and the Gradual Audiomotor Evolution hypothesis (Merchant and Honing, 2014) and review recent experimental progress supporting each of these hypotheses. While initial formulations of ASAP and GAE explain different aspects of beat-based timing-the involvement of motor structures in the absence of movement, and physical entrainment to an auditory beat respectively-we suggest that work under both hypotheses provide converging evidence toward understanding the predictive role of the motor system in the perception of rhythm, and the specific neural mechanisms involved. We discuss future experimental work necessary to further evaluate the causal neural mechanisms underlying beat and rhythm perception.

Cover page: Motor and Predictive Processes in Auditory Beat and Rhythm Perception

Article
Peer Reviewed

Investigating the role of auditory cues in modulating motor timing: insights from EEG and deep learning

UC Merced Previously Published Works (2024)

Research on action-based timing has shed light on the temporal dynamics of sensorimotor coordination. This study investigates the neural mechanisms underlying action-based timing, particularly during finger-tapping tasks involving synchronized and syncopated patterns. Twelve healthy participants completed a continuation task, alternating between tapping in time with an auditory metronome (pacing) and continuing without it (continuation). Electroencephalography data were collected to explore how neural activity changes across these coordination modes and phases. We applied deep learning methods to classify single-trial electroencephalography data and predict behavioral timing conditions. Results showed significant classification accuracy for distinguishing between pacing and continuation phases, particularly during the presence of auditory cues, emphasizing the role of auditory input in motor timing. However, when auditory components were removed from the electroencephalography data, the differentiation between phases became inconclusive. Mean accuracy asynchrony, a measure of timing error, emerged as a superior predictor of performance variability compared to inter-response interval. These findings highlight the importance of auditory cues in modulating motor timing behaviors and present the challenges of isolating motor activation in the absence of auditory stimuli. Our study offers new insights into the neural dynamics of motor timing and demonstrates the utility of deep learning in analyzing single-trial electroencephalography data.

Cover page: Investigating the role of auditory cues in modulating motor timing: insights from EEG and deep learning