Object recognition is of essential importance to the ways in which humans explore and interact with objects in the world by sight and touch. Though visual and tactile information are sensed independently, it appears that certain brain areas, namely the lateral occipital complex, perirhinal cortex, and anterior intraparietal sulcus, may integrate visual and haptic information to create an object representation. A pressing question in this field is how these brain areas integrate and compare visual and haptic information in order to recognize objects we are seeing and feeling as the ‘same’ or ‘different.’ There are many different complex hypotheses for how this comparison across modalities occurs. In this study, our goal was to take a first step towards elucidating this complex process of visuo-haptic transfer and comparison of information across modalities in the human brain. We have begun to answer these guiding questions by creating novel visual and haptic stimuli to be used for our proposed experimental task, a two-alternative forced choice visual and haptic shape comparison task within and across modalities; training a novel neural network to track human hand grasp position to study the role of grasp pre-shaping in visuo-haptic shape comparison; and utilizing an eye tracker to study the effects of shape comparison within and across modalities on eye position. After the development of these novel methods, we tested their function through a small pilot study. Our results provide insight into the function of these methods and their potential for studying the speed and accuracy of cross-modal learning of novel shapes and corresponding changes in eye and grasp position in human participants. Future directions will include expanding the sample size in order to elucidate this complex process of transfer and comparison of object information across modalities in the human brain.
Kasi, Anisha,
"Within-Modal and Cross-Modal Learning and Recognition of Shapes through Vision and Touch in Humans"
(2023).
Neuroscience Theses and Dissertations.
Brown Digital Repository. Brown University Library.
https://doi.org/10.26300/rqcw-f320
