Description
- Abstract:
- Real-world events produce both auditory and visual signals that, together, comprise our perceptual experience of that event. At issue is how the brain integrates information from these independent perceptual channels to form coherent percepts. Low-level factors, such as common temporal and spatial structure (Meredith and Stein, 1993), and high-level factors, such as semantic labeling (Doehrmann and Naumer, 2008), both appear to influence integration. In a series of three studies, we examined how two of these integration cues, temporal synchrony and semantic congruency, interact with one another during the neural integration of audiovisual (AV) events. We generated a novel stimulus set of audio-visual movies of environmental events, such as tapping a pencil or cutting paper. These movies allowed us to manipulate congruency between the temporal structure of the auditory and visual signals independently from the semantic labeling of the same auditory and visual signals. The first two studies, employing functional neuroimaging (fMRI), identified separable networks of regions for each of the integration cues, including a preference in the right hemisphere for time and a preference in the left hemisphere for semantics. The third study, employing EEG, complemented the fMRI findings by finding that each of the integration cues influenced the integration process at different time windows. The effects of temporally synchrony preceded the effects of semantic congruency. Together, these three studies effectively isolated the separable effects of both signal-based (time) and high-level (semantic) integration cues, finding that these two cues rely on functionally specialized neural substrates as well as different timescales during the process of multimodal integration. By identifying the separable components of the multimodal processing network, our study advances the understanding of how modality-specific information is bound into coherent event percepts.
- Notes:
- Thesis (Ph.D.) -- Brown University (2010)
Access Conditions
- Rights
- In Copyright
- Restrictions on Use
- Collection is open for research.
Citation
Vettel, Jean M.,
"NEURAL INTEGRATION OF MULTIMODAL EVENTS"
(2009).
Cognitive Sciences Theses and Dissertations.
Brown Digital Repository. Brown University Library.
https://doi.org/10.7301/Z0VQ30ZR
Relations
Collection:
-
Cognitive Sciences Theses and Dissertations
Theses and Dissertations for the Cognitive Sciences department....