NEURAL INTEGRATION OF MULTIMODAL EVENTS Vettel, Jean M. creator 2009 xiii, 128 p. born digital Thesis (Ph.D.) -- Brown University (2010) Tarr, Michael director Blumstein, Sheila reader Heller, Laurie reader Sheinberg, David reader Brown University. Cognitive and Linguistic Sciences: Cognitive Sciences sponsor theses Real-world events produce both auditory and visual signals that, together, comprise our perceptual experience of that event. At issue is how the brain integrates information from these independent perceptual channels to form coherent percepts. Low-level factors, such as common temporal and spatial structure (Meredith and Stein, 1993), and high-level factors, such as semantic labeling (Doehrmann and Naumer, 2008), both appear to influence integration. In a series of three studies, we examined how two of these integration cues, temporal synchrony and semantic congruency, interact with one another during the neural integration of audiovisual (AV) events. We generated a novel stimulus set of audio-visual movies of environmental events, such as tapping a pencil or cutting paper. These movies allowed us to manipulate congruency between the temporal structure of the auditory and visual signals independently from the semantic labeling of the same auditory and visual signals. The first two studies, employing functional neuroimaging (fMRI), identified separable networks of regions for each of the integration cues, including a preference in the right hemisphere for time and a preference in the left hemisphere for semantics. The third study, employing EEG, complemented the fMRI findings by finding that each of the integration cues influenced the integration process at different time windows. The effects of temporally synchrony preceded the effects of semantic congruency. Together, these three studies effectively isolated the separable effects of both signal-based (time) and high-level (semantic) integration cues, finding that these two cues rely on functionally specialized neural substrates as well as different timescales during the process of multimodal integration. By identifying the separable components of the multimodal processing network, our study advances the understanding of how modality-specific information is bound into coherent event percepts. multimodal audio-visual FMRI EEG TimeSemantics RPB 20091218 engEnglish10.7301/Z0VQ30ZRIn CopyrightCollection is open for research.dissertations