Transcript Function Magnetic Resonance Imaging (fMRI):
Multisensory Integration in the Brain
Paul J. Laurienti, M.D., Ph.D.
Department of Radiology
McGurk Illusion
http://www.wam.umd.edu/~vvw/McGurk.htm
Why Multisensory?
1.
2.
Survey of Language Knowledge LOUISA C. MOATS, Ed.D VALERIE G. TUCKER, M.Ed., CALT President, The Alliance for Accreditation and Certification of Multisensory Structured Language Education.
3.
4.
5.
6.
7.
8.
9.
Panel of Experts Moderator: VALERIE G. TUCKER, M.Ed. A panel of experts will address the National Reading Panel Competencies and give teachers practical ideas for implementing multisensory structured activities for each identified area.
Capturing Elusive Images: Practical Strategies for Building Orthographic Recall MICHELE BERG, Ph.D
Teaching the Teachers: Effective Models for Colleges and Universities Dyslexia and the English Language Learner ELSA CÁRDENAS-HAGAN, Ed.D, Beyond Basic Phonics: The MultiSyllabic “Missing Link” to Literacy POLLY BAYRD, M.A and KRISTIN JOHNSON, M.A.
Strengthening Vocabulary with Direct Instruction and Morphology BRYAN K. CLAYBACK, B.A. et al.
Expressive Phonological Impairment and Spelling: Impact of a Multisensory Intervention DEBORAH USREY, Ed.D
10. Teaching Non-Literate and Brain-Injured Adults Successfully with Sounds in Syllables JENNIE LYNCH, B.A. and SANDRA DILLON, M.A.
11. Effects of a Multisensory-Based Comprehensive Reading District Reform Model PAUL WORTHINGTON 12. The ABCs of Grammar Instruction: Keeping it Structured, Sequential and Multisensory PHYLLIS C. HUTSON 13. Hands-On, MultiSensory Reading Instruction GERI MARSHALL MOHLER, Ed.D. and BETH ANN BADER-PAETSCHOW, Ed.D.
14. The Association Method: Successful Strategies for Oral and Written Language Intervention MAUREEN K. MARTIN, Ph.D.
15. Make it Fun! Make it Challenging! Make it Multisensory! SUE GNAGY FEGAN, M.S., F/AOGPE and S C OTT MURRILL, M.A.
16. Considerations of Private Practice ELLEN J. LINSKY, Ed.D., M/AOGP 17. Implementing Undergraduate and Graduate Multisensory Structured Language Teacher Training Programs: Two Examples KEICHER and SCHNEIDER 18. The Student Notebook: An Important Component of Multisensory Structured Language Instruction. KAREN K. LEOPOLD, M.Ed.
19. The Roots of Algebra Revealed: Concepts Before Procedures MARILYN ZECHER, M.A., C.A.T.
20. Preparing Preschool Children to Read: A Randomized Trial Study MARCIA DAVIDSON, Ph.D., JENNAE BULAT, M.A., M.B.A., and LAUREL COCO, M.A.
21. Part B: The Painted Essay: A Multisensory Approach for Essay Writing and Assessment CARLENE BADEAU, B.A.
22. Part A: Reading Remediation in the Middle School Through Explicit Phonics Instruction GERI MARSHALL MOHLER, Ed.D
23. The Slingerland Multisensory Approach to Reading BONITA COLONY and SUE WINTERS-BROW N, M.Ed.
24. Teaching the Teachers: Effective Models for Colleges and Universities MAUREEN K. MARTIN, Ph.D. et al 25. Fun and Games in an OG Lesson COREY ZYLSTRA 26. Does Multisensory Instruction Meet the Criteria for Scientifically-Based Reading Research (SBRR)? DEBORAH F. KNIGHT, Ph.D. and MARGIE GILLIS, Ph.D
27. Building Bridges Through Collaboration: Reading Disabilities Prevention and Intervention DEBORAH MIDDEL–KAT Z E N M E Y E R, M.A. et al 28. Prevention and Early Intervention: Multisensory Instruction in K-3 General Education Classrooms ROBIN CARLO, M.A
29. Slingerland Multisensory Approach for Upper Elementary through High School Students S U SAN HEINZ, Ph.D.
Why Multisensory?
We live in a “multisensory” world
• • •
Information can be processed on a modality specific basis Information converges and is integrated in the brain Performance enhancement is larger for multisensory than unisensory stimuli
Color Discrimination Task
(Multisensory) (Unisensory)
Multisensory Enhancement
Cross-modal
700 600 500 400 300 200 100 0
*
Auditory Visual Multisensory
Dual-visual ns
Visual 1 Visual 2 Dual-Visual 700 600 500 400 300 200 100 0
Factors Modulating Multisensory Integration
•
Temporal/spatial coincidence
• •
Semantic congruence Sensory attention
–
Selective
–
Divided
Temporal Coincidence
+
Hairston et al. Exp Brain Res (2005) 166: 474–480
Temporal Coincidence
+
Hairston et al. Exp Brain Res (2005) 166: 474–480
Temporal Coincidence
Hairston et al. Exp Brain Res (2005) 166: 474–480
Factors Modulating Multisensory Integration
•
Temporal/spatial coincidence
• •
Semantic congruence Sensory attention
–
Selective
–
Divided
Semantic Congruence
Visual Multisensory Congruent Multisensory Incongruent
Factors Modulating Multisensory Integration
•
Temporal/spatial coincidence
• •
Semantic congruence Sensory attention
–
Selective
–
Divided
Selective Attention – Multisensory Processing
2.2
2 1.8
1.6
divided attend_vis attend_aud integration 1.4
1.2
1 0.8
0.6
300 350 400 450 500 550 600
RT (ms)
650 700 750 800 850
Selective Attention – Multisensory Processing
•
Selective attention suppresses multisensory integration Unpublished data removed
Behavioral studies
•
Redundant multisensory benefits >> redundant unisensory benefits
•
Multisensory Integration is modulated stimulus features
–
Timing
–
Location
–
Meaning
•
Congruent – enhancements
•
Incongruent- decrements
•
Sensory attention modulates multisensory integration
Multisensory Brain Regions
Brodmann Areas
• • • •
Superior temporal sulcus
–
STS (BA 22) Inferior temporo-occipital
–
BA 19/37 Angular gyrus
–
BA 39/40 Cingulate gyrus
–
BA 32/9/10 http://spot.colorado.edu/~dubin/talks/brodmann/brodmann.html
Superior Temporal Sulcus
Beauchamp et al. – Nature Neuroscience 2004, 1190-1192 Calvert et al, Current Biology 2000, 10:649–657 Atteveldt et al. Neuron 2004 43, 271–282.
Inferior Temporo-occipital
Hashimoto et al, Neuron, Vol. 42, (2004) 311–322 L. Cohen et al. NeuroImage 23 (2004) 1256–1270 Beauchamp et al, Neuron, Vol. 41, 809–823, March 4, 2004,
Angular and Cingulate Gyri
Linguistic de Zubicaray et al. (2002) Saito et al Cereb Cortex. 2005, 15:1750-60 Nonlinguistic Laurienti et al. (2003)
Unisensory Brain Regions
• • •
Do the unisensory cortices interact?
How does attention modulate unisensory activity?
Are unisensory interactions altered in dyslexia?
Brodmann Areas http://spot.colorado.edu/~dubin/talks/brodmann/brodmann.html
Vision
Unisensory Brain Regions
Audition
•
Cross-modal deactivations are modulated by selective attention
Laurienti et al. (2002)
Unpublished data removed
Phoneme Recognition Task
•
Dyslexic readers do not deactivate visual cortex during an auditory processing task Unpublished data removed
Imaging Multisensory Processes
•
Multiple multisensory cortical regions
•
Factors that modulate behavior (coincidence, congruence, attention) also modulate fMRI
•
Unisensory and multisensory regions are closely situated
•
Cross-modal deactivations
•
Altered multisensory processing in dyslexia