Shinsuke Shimojo

Integration Beyond Sensory Modalities

California Institute of Technology

Although brain localization and computational modularity of sensory modality are often claimed (sometimes even with regard to "innateness" of it), they tend to be overemphasized. Literature on neural plasticity of sensory systems in young animals and deprived humans, for example, is an abundant source for evidence against fixed brain localization and modularity. How much do we know, however, about perceptual integration across modalities in the normal adult humans? Our latest psychophysical findings also dispute localization/modularity in the strict sense. We have found that: (a) sounds can either improve, or degrade temporal order judgment in vision, depending on temporal relationship (1); (b) a sound (or a transient signal in visual or tactile modality) that is synchronized with visual crossing of two objects enhances bounce (collision) and suppresses streaming perception (2); and (c) a single flash can be seen as multiple flashes if it is accompanied with multiple sounds (3). These findings argue against the classical view of the human as a vision-dominated animal, and for crossmodal modulation at early cortical levels and transient signal as the key for perceptual integration. Our developmental study of the stream/bounce perception ((b) above) in human infants further indicates that infants first show behavioral evidence of adult-like categorical perception of a visual motion event modulated by a sound only after the age of 5-6 months, and that it is most likely constrained by development of spatial attention (4, 5). Perhaps the most classical view of crossmodal integration in the human was that it is primarily a vision-dominated animal. Due to a line of evidence indicating auditory modulation of vision in temporal domain, a modern "modality precision (appropriateness)" hypothesis was proposed. Our view yet differs from this; we believe that transient sensory signal is critical in crossmodal integration, regardless of whether the transient signal comes within the modality or across modalities (6).

References
(1) Scheier. C., Nijhawan, R., and Shimojo, S. (1999) Invest. Ophthal. Vis. Sci., 40(4), 4169.
(2) Watanabe, K. and Shimojo, S. (1998) Perception, 27, 1041-.
(3) Shams, L., Kamitani, Y. and Shimojo, S. (2000) Nature, 408, 788.
(4) Scheier, C., Lewkowicz, D., and Shimojo, S. (2000) Invest. Ophthal. Vis. Sci., 41(4), S327.
(5) Shimojo, S., Watanabe, K., and Scheier, C. In: Visual Attention and Cortical Circuits. J. Braun, C. Koch, and J. Davids (eds.), MIT Press (in press).
(6) Shimojo, S., and Shams, L. Sensory Modalities are Not Separate Modalities: Plasticity and Interaction Beyond Modalities. Current Opinion in Neurobiology, (in press).

Friday, February 23, 2024
About the Swartz Foundation...
 
The Swartz Foundation was established by Jerry Swartz (bio) in 1994 . . .
more>
 
Follow us...
 
The Swartz Foundation is on Twitter: SwartzCompNeuro
more>
 
 
2013 Stony Brook Mind/Brain Lecture - Michael Wigler, PhD
 
 
2012 Stony Brook Mind/Brain Lecture - John Donoghue
 
 
Sloan-Swartz Centers Annual Meeting 2011
 
 
2011 Stony Brook Mind/Brain Lecture - Allison J. Doupe
 
 
2011 Banbury Workshop
 
 
Sloan-Swartz Centers Annual Meeting 2010
 
 
2010 Stony Brook Mind/Brain Lecture
 
 
Sloan-Swartz Centers Annual Meeting 2009
 
 
Conference on Neural Dynamics
 
 
2009 Stony Brook Mind/Brain Lecture
 
 
Canonical Neural Computation, April 2009
 
 
2009 Banbury Workshop
 
 
Sloan-Swartz Centers Annual Meeting 2008
 
 
Theoretical and Experimental Approaches to Auditory and Visual Attention - Banbury 2008
 
 
Stony Brook Mind/Brain 2008: Patricia Smith Churchland, B. Phil. D
 
 
Sloan-Swartz Centers Annual Meeting 2007
 
 
New Frontiers In Studies Of Nonconscious Processing - Banbury 2007
 
 
Stony Brook Mind/Brain 2007: Professor Michael Shadlen, MD, PhD
 
 
Multi-level Brain Modeling Workshop 2006
 
 
Sloan Swartz Centers Annual Meeting 2006
 
 
Banbury 2006: Computational Approaches to Cortical Functions
 
 
Stony Brook Mind/Brain 2006: Helen Fisher -- Lecture Videos
 
 
Sloan-Swartz Centers for Theoretical Neurobiology
 
 
Swartz Center for Computational Neuroscience
 
 
Banbury Center Workshop Series
 
 
Other Events
 
www.theswartzfoundation.org                           Copyright © The Swartz Foundation 2024