- Title Information
- Title
- NEURAL INTEGRATION OF MULTIMODAL EVENTS
- Name:
Personal
- Name Part
- Vettel, Jean M.
- Role
- Role Term:
Text
- creator
- Origin Information
- Copyright Date
(keyDate="yes", encoding="w3cdtf")
- 2009
- Physical Description
- Extent
- xiii, 128 p.
- digitalOrigin
- born digital
- Note
- Thesis (Ph.D.) -- Brown University (2010)
- Name:
Personal
- Name Part
- Tarr, Michael
- Role
- Role Term:
Text
- director
- Name:
Personal
- Name Part
- Blumstein, Sheila
- Role
- Role Term:
Text
- reader
- Name:
Personal
- Name Part
- Heller, Laurie
- Role
- Role Term:
Text
- reader
- Name:
Personal
- Name Part
- Sheinberg, David
- Role
- Role Term:
Text
- reader
- Name:
Corporate
- Name Part
- Brown University. Cognitive and Linguistic Sciences: Cognitive Sciences
- Role
- Role Term:
Text
- sponsor
- Genre (aat)
- theses
- Abstract
- Real-world events produce both auditory and visual signals that, together, comprise our perceptual experience of that event. At issue is how the brain integrates information from
these independent perceptual channels to form coherent percepts. Low-level factors, such as common temporal and spatial structure (Meredith and Stein, 1993), and high-level factors, such as
semantic labeling (Doehrmann and Naumer, 2008), both appear to influence integration. In a series of three studies, we examined how two of these integration cues, temporal synchrony and semantic
congruency, interact with one another during the neural integration of audiovisual (AV) events. We generated a novel stimulus set of audio-visual movies of environmental events, such as tapping
a pencil or cutting paper. These movies allowed us to manipulate congruency between the temporal structure of the auditory and visual signals independently from the semantic labeling of the same
auditory and visual signals. The first two studies, employing functional neuroimaging (fMRI), identified separable networks of regions for each of the integration cues, including a preference in
the right hemisphere for time and a preference in the left hemisphere for semantics. The third study, employing EEG, complemented the fMRI findings by finding that each of the integration cues
influenced the integration process at different time windows. The effects of temporally synchrony preceded the effects of semantic congruency. Together, these three studies effectively isolated
the separable effects of both signal-based (time) and high-level (semantic) integration cues, finding that these two cues rely on functionally specialized neural substrates as well as different
timescales during the process of multimodal integration. By identifying the separable components of the multimodal processing network, our study advances the understanding of how
modality-specific information is bound into coherent event percepts.
- Subject (Local)
- Topic
- multimodal
- Subject (Local)
- Topic
- audio-visual
- Subject (Local)
- Topic
- FMRI
- Subject (Local)
- Topic
- EEG
- Subject (FAST)
(authorityURI="http://id.worldcat.org/fast", valueURI="http://id.worldcat.org/fast/1151043")
- Topic
- Time
- Subject (FAST)
(authorityURI="http://id.worldcat.org/fast", valueURI="http://id.worldcat.org/fast/1112079")
- Topic
- Semantics
- Record Information
- Record Content Source (marcorg)
- RPB
- Record Creation Date
(encoding="iso8601")
- 20091218
- Language
- Language Term:
Code (ISO639-2B)
- eng
- Language Term:
Text
- English
- Identifier:
DOI
- 10.7301/Z0VQ30ZR
- Access Condition:
rights statement
(href="http://rightsstatements.org/vocab/InC/1.0/")
- In Copyright
- Access Condition:
restriction on access
- Collection is open for research.
- Type of Resource (primo)
- dissertations