Chapter 4: Cognitive science and the integration challenge

What is cognitive science?

• Is there such a thing as cognitive science (as opposed to the cognitive sciences)?

• Are there any basic assumptions about the mind that unify all the different models and approaches we have been considering?

• Does cognitive science have a distinctive methodology?

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

An early model

• Identifies connections between different cognitive sciences, but sheds no light on why there need to be such connections • Doesn’t help us to understand what each discipline contributes • Doesn’t give a very accurate picture of contemporary cognitive science

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Two basic assumptions

(1) The mind is too complicated to be understood using the resources of any discipline on its own •  Need for interdisciplinarity (2) The mind has to be understood at many different levels  Need for an integrated multilevel explanation •

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

The example of neuroscience

Neuroscience has evolved enormously since the early days of cognitive science It is itself highly interdisciplinary and multilevel Illustrates the close connection between • levels of organization • levels of explanation • available scientific tools and techniques

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Levels of explanation A B C D E-H Experimental psychology Cognitive neuroscience Behavioral neuroscience Systems neuroscience Cellular neuroscience Molecular neuroscience

Tools and levels in neuroscience

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Organization in psychology

• Scientific psychology is not organized vertically in terms of levels • Instead organized horizontally in terms of domains MIND AND BEHAVIOR AT THE LEVEL OF THE ORGANISM General cognitive capacities COGNITIVE PSYCHOLOGY Cognition in a social context SOCIAL PSYCHOLOGY Individual personality and character PERSONALITY PSYCHOLOGY Non-human cognition COMPARATIVE PSYCHOLOGY How cognitive abilities develop DEVELOPMENTAL PSYCHOLOGY

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

The integration challenge

• The challenge of providing an unified account of cognition that draws upon and integrates the whole space • Many regions within the “space” of cognitive science remain little studied • The “space” is not organized by discipline

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

3 approaches to IC

Local integrations • Examples of specific cases where cognitive scientists have built bridges across levels of explanation and between disciplines Global models of integration • Blueprints for solving the integration challenge The mental architectures approach

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Local integration 1

• Inspired by experiments in the psychology of reasoning showing that people are systematically prone to very specific types of fallacies in deductive reasoning • Evolutionary psychologists suggest that these patterns of error are by-products of highly specialized cognitive mechanisms that emerged to solve adaptive problems • massive modularity thesis • cheater detection module

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Local integration 1

• Solution of adaptive problems

Explains

• Emergence of dedicated cheater detection system

Explains

• Patterns of error in logical reasoning tasks

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Local integration 2

• Cognitive scientists often treat functional neuroimaging as if it provides direct “map” of the mind in action • But neuroimaging directly measures blood flow (PET) or levels of blood oxygen (fMRI) – whereas the “motor” of cognitive activity is neuronal activity • How do we integrate BOLD signal information with information about neuronal activity?

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Local integration 2

• Research question: What are the neural correlates of the BOLD signal?

• Experiments by Logothetis explored this question by calibrating fMRI data with microelectrode recording • bridging levels of organization and levels of spatial resolution • allowed fMRI data to connect up with neural models (and computational neuroscience)

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Models of global integration

Two main candidates: • Marr’s tri-level hypothesis - explicitly proposed as a way of bridging different levels of explanation 1. computational level 2. algorithmic/representational level 3. physical level • Models of inter-theoretic reduction derived from philosophy of science – analogy with unity of science hypothesis in the physical sciences

Cognitive Science



José Luis Bermúdez / Cambridge University Press 2010

Psychology of reasoning

• Psychologists have produced evidence that subjects regularly contravene basic principles of deductive logic probability theory when engaged in • conditional reasoning • judgments of likelihood

Conditional reasoning rules

Modus ponens If

p

then

q p

Therefore

q

Affirming the consequent If

p

then

q q

Therefore

p

Modus tollens If

p

then

q

not-

q

Therefore not-

p

Denying antecedent If

p

then

q

not-

p

Therefore not-

q

Wason selection task

What cards need to be turned over to evaluate: If a card has a vowel on one side then it has an odd number on the other side

Cassava root studies (Cosmides and Tooby)

Background about (imaginary) Pacific island: • Only married men have facial tattoos • Cassava roots are a highly prized delicacy and aphrodisiac • Molo nuts are bitter and not valued in the community

Social exchange

Two versions of cassava root story

Descriptive

: married men live on the side of the island where cassava roots grow, while unmarried men live on the side where the molo nuts grow

Social exchange

: only married men have the right to eat cassava roots.

Test and results

• If a man is eating cassava root, he must have a tattooed face EATS CASSAVA ROOT EATS MOLO NUTS TATTOO NO TATTOO Descriptive version Poor performance (21%) Social exchange version Better performance (75%)

Cosmides and Tooby analysis

• EATS CASSAVA ROOT Benefit EATS MOLO NUTS No benefit TATTOO Cost paid NO TATTOO Cost not paid Social exchange version has following structure If BENEFIT then COST Cheater = BENEFIT without COST [i.e.

p

& ~

q

]

Local integration 1

• Solution of adaptive problems

Explains

• Emergence of dedicated cheater detection system

Explains

• Patterns of error in logical reasoning tasks

The structure of the argument!!

• If CONDITIONAL REASONING EXPLOITS A CHEATER DETECTION MODULE (

p

) then PERFORMANCE WILL BE BETTER ON THE SOCIAL EXCHANGE VERSION (

q

) • PERFORMANCE IS BETTER ON THE SOCIAL EXCHANGE VERSION (

q

) • Therefore, CONDITIONAL REASONING EXPLOITS A CHEATER DETECTION MODULE (

p

)

Switched selection task

Standard social exchange selection task • If BENEFIT (

p

) then COST (

q

) • violation =

p

and not-

q

Switched social exchange selection task • If COST (

p

) then BENEFIT (

q

) • violation =

q

and not-

p

• Subjects typically give the logically correct answer on the standard version, but not on the switched version • Detecting a violation of the switched version is not the same as detecting a counter-example to the conditional

Cosmides and Tooby analysis

• EATS CASSAVA ROOT Benefit EATS MOLO NUTS No benefit TATTOO Cost paid NO TATTOO Cost not paid Switched social exchange version has following structure If COST then BENEFIT Logically correct answers are cards 2 and 3 Cheater detection answers remain 1 and 4

Evolutionary psychology and conditional reasoning

• evolutionary psychologists reject the idea of domain-general reasoning skills either mental logic or mental models • suggest that we employ context-dependent inference rules – in particular, rules for detecting cheaters in social exchanges • integrate these experimental data with a model of how the mind is organized and how it evolved

Massive modularity thesis

Gives a picture of the overall organization of the mind • mind composed of highly specialized cognitive modules (Darwinian modules) • each module evolved to solve a particular adaptive problem • each module exploits specialized rules that are

domain-specific •

No domain general “central cognition” or abstract reasoning mechanism

Cheater detection module

The Cosmides/Tooby experiments seem to show specialized skills for cheater detection • not simply specialized skills for conditional reasoning involving social exchanges These experimental results are integrated with the massive modularity hypothesis via an evolutionary explanation of why there needs to be a cheater detection module • evolutionary explanation itself grounded in an account of the evolution of altruism

The puzzle of altruistic behavior

• Cooperative behavior widespread in animal kingdom • even in lower animals ants, termites, bees etc (individuals fed by others etc) • not restricted to kin • Cooperative behavior presumably has a genetic basis • But how did the genes coding for cooperative behavior ever get established in the gene pool?

• natural selection seems to favor “selfish” behavior - free riders can always exploit altruists

Modeling the evolution of cooperation

The prisoner’s dilemma is a very useful tool for modeling the problem • we can assume that participants are purely selfish • set up so that cooperation is not the dominant strategy for • can easily be extrapolated to multi-person interactions (tragedy of the commons)

One-shot PD

• • •

Player

• •

B

COOP DEFECT COOP 5, 5 0, 10

Player A

DEFECT 10, 0 2, 2 • Illustrates basic structure of interactions where being a free rider is advantageous

Decision-making in a one-shot PD

• Work backwards from what the other agent might do • Look at your options if the other agent cooperates – it is best for you to defect • Look at your options if the other agent defects – it is best for you to defect • The dominant strategy for each play is DEFECT • But mutual defection is sub-optimal

Iterated PDs

• A backwards induction argument shows that DEFECT is dominant when the number of plays is known • But for modeling the evolution of cooperation the interesting case is the indefinitely iterated PD • opens up possibility of strategies that “punish” other player for defecting • and “rewarding” for cooperating

Axelrod’s computer tournament

Invited game theorists to submit strategies for iterated PD tournament • played strategies against each other for around 200 iterations Highest average score came from TIT-FOR-TAT • Start by cooperating • Then do what the opponent did on the previous round

TIT-FOR-TAT

• Shows how cooperative behavior might emerge in very simple organisms • and be maintained since, in the right conditions, it is an evolutionarily stable strategy • Some evidence that TIT-FOR-TAT is followed in the animal kingdom (3-spined sticklebacks) • Has been used to model complicated human interactions (e.g. voting patterns in US Senate)

Back to cheater detection

• TIT-FOR-TAT (or some similar strategy, such as TIT FOR-TWO-TATS) can only work if there is a reliable mechanism for detecting cheaters. . .

• Evolutionary pressure for selection of cheater detection module • According to Cosmides and Tooby, this module explains the pattern of choices made in conditional reasoning tasks

Local integration 1

• Solution of adaptive problems

Explains

• Emergence of dedicated cheater detection system

Explains

• Patterns of error in logical reasoning tasks

4 0

언어처리 의 인지신경과학적 접근

• 언어처리

(language processing)

란

?

– 정보처리적 (informational processing) 관점에서 , 산출하고 , 이해하는 과정을 지칭하는 용어 – 사람의 말과 글을 말

(

말소리

)(speech, vocal output):

사람의 성대와 구강을 통해 조음되어 나오는 의미 있는 소리 – 글

(text)

: 말소리가 일정한 표기체계 ( writing system) 를 통해 표현된 물리적 실체 – 말소리 또는 글을 산출하거나 이해하는 과정이 어떤 단계로 구성될 수 있으며 , 그리고 그 단계들이 어떻게 유기적으로 상호작용하여 최종적인 산출이나 이해에 도달하게 되는지를 연구함 .

• 예 ) 음소지각 음운표상 / 처리 단어표상 / 처리 문장구성 / 처리 담화 / 덩이글 표상과 처리

4 1 언어처리

의

•

인지신경과학적 접근

인지신경과학

(Cognitive Neuroscience)

이란 ?

– – 인지과학

(Cognitive Science)

의 중심학문 인지심리학 : 인간의 마음의 작용을 자극 반응 (stimulus-response) 의 조건형성 (conditioning) 만으로도 충분히 설명할 수 있다는 행동주의 (behaviorism) 심리학을 넘어서 , 기억 / 주의 (attention) 언어처리 등과 같은 인지과정이 마음의 중추적인 작용이라 상정하고 , 이를 정보처리적인 모델로서 구현하고 , 연구실에서 이루어지는 행동실험 컴퓨터 시뮬레이션 등을 통해 이를 검증하는 학문 / – 신경과학 : 신경세포 (neuron) 의 정보전달체계를 연구하는 학문으로 작게는 단일 뉴런의 생성과 정보처리 , 크게는 중추신경계의 구조적 연결과 기능을 연구하며 , 20 세기에 들어 중추신경계 구조가 연구의 초점 ( 뇌 ) 의 기능과 – 인지심리학과 신경과학의 결합 • 인지과정을 처리하는 뇌의 기능과 구조를 연구함 • 비침습적 (non-invasive) 뇌기능 연구 방법론의 발달 • 21 세기는 뇌 과학의 시대

4 2

인지신경과학적 연구방법들

• 침습적 (invasive) 방법들

field potential(LFP)

4 3

인지신경과학적 연구방법들

• 침습적 (invasive) 방법들 – George Ojemann (1989)’s direct cortical stimulation during brain surgery. – 간질환자의 간질부위 절개시술 전 언어영역을 찾기 위해 전기막대로 언어영역을 자극함 – 뇌 국소마취 , 환자는 깨어있음 그림을 보여주고 , 그림의 이름을 말하게 함

4 4

인지신경과학적 연구방법들

• 비침습적 (non-invasive) 연구방법들 – TMS(Transcortical Magnetic Stimulation)

4 5

인지신경과학적 연구방법들

• 비침습적 (non-invasive) 연구방법들 – EEG(Electroencephalogram), ERP(event-related potential), – MEG(Magnetoencephalogram)

4 6

인지신경과학적 연구방법들

• 비침습적 (non-invasive) 연구방법 – Neuroimaging 방법의 대두 – PET(positron emission topography)

4 7

신경과학적 연구방법들

• 비침습적 (non-invasive) 연구방법 – Neuroimaging 방법의 대두 – fMRI(functional Magnetic Resonance Imaging)

4 8

뇌와 언어

• • Paul Broca (1861), French physician – 언어산출에 장애를 보이는 두 명의 환자의 뇌를 사후 해부하여 , 좌하전두엽이 언어산출에 관여함을 주장함 .

– 오른쪽 뇌의 유사부위를 다친 사람은 언어산출에는 크게 장애를 보이지 않음 – 좌하전두엽을 Broca’s region( 브로카 영역 ) 이라 일컬음 .

Carl Wernicke (1884), German psychiatrist – 왼쪽 상/중측두엽 - 언어이해

4 9

실어증 환자 연구

• • • Broca’s aphasia: – – – – 언어산출의 장애(기능어(전치사, 조사), 동사 태/시제 등의 손실) 심각한 말더듬, 내용어 중심의 산출 문법구조가 복잡한 문장이해의 장애 • Reversible sentences – picture matching) • “ The white dog was chased by the brown horse.” 실험지시문을 이해 Wernicke’s aphasia – 언어이해의 장애 – – 의미 없는 말의 나열(word salad), 기본적인 통사구조는 유지 실험지시문을 이해하지 못함 Conduction aphasia(전도성 실어증) – Broca 영역과 Wernicke 영역을 잇는 뇌섬유다발(Arcuate Fasciculus)의 손상 – 자발적 산출과 이해에 크게 손상은 없으나, 따라말하기 과제에 특수한 장애를 보임

5 0

Wernicke-Geshwind model - 1965

언어산출 / 통사처리 Motor Cortex Arcuate Fasciculus 음운표상처리 Supramarginal Gyrus Angular Gyrus 음운표상 철자표상 전환 Broca’s Area 청각적 단어 재인 Auditory Cortex Wernicke’s Area 언어이해 / 의미처리 시각적 단어 재인 Visual Cortex

5 1

실어증 연구의 한계

• • 뇌손상 부위의 다양성 – 같은 장애를 보이는 환자들의 뇌손상부위가 다양하며 , 비슷한 뇌손상 부위를 가져도 다른 장애를 보이는 경우가 많음 – 정확한 뇌영역의 위치와 기능을 상세히 규명하기 어려움 뇌손상 원인의 다양성 – 물리적 외상 / 뇌경색 / 치매 등 다양한 원인 – 발병시기의 다양성 여러 변인들을 통제할 수 있는 인지신경과학적 실험적 방법들을 필요로 함 .

What is the functional MRI?

• MRI(Magnetic Resonance Imaging) + a fast imaging technique Statistical Parametric Map fMRI time-series

3T magnet RF Coil gradient coil (inside)

• • • • • 신경세포의 활동은 뇌의 국소적인 에너지 대사량을 증가시킴 에너지 대사에 필요한 산소는 헤모글로빈에 의해 전달됨 산소를 빼앗긴 헤모글로빈은 자기장에 영향을 주는 상자성 물질 MRI 이미지는 뇌의 구성물질이 자기장의 영향을 주는 정도를 표현함 .

따라서 , 신경세포 활동에 따른 국소적인 에너지 대사를 나타내는 산소소모량의 변화의 시간적 추이를 측정하는 방법 5 2

Neural activity



BOLD signal

stimulatio n Psychologi cal variable

Task Direct measurements Single cell recording Local field potential Multi-units activity 5 EEG(ERP), MEG 3 Behavioral response Neuronal response Task Linear transformation BOLD signal Hemodynamic response fMRI, PET

5 4

뇌영상 방법을 통한 언어처리 연구의 실제

• •

Lee and Newman(2009), Human brain mapping The Effect of Presentation Paradigm on Syntactic Processing: An Event-Related fMRI Study

– 문장 제시 방법을 조절하여 문장처리와 기억과의 관계 분석 – 문장의 통사적 난이도 조절(평서연결문 vs. 목적관계절 삽입문) • The pilot scared the escort and broke the mirror on the closet.

• The pilot that the escort scared broke the mirror on the closet.

– 목적관계절 삽입문은 보다 많은 작업기억을 요구함. => 문장의 즉시적 처리의 어려움 – 문장 제시 방법의 차이(전체 제시, 한단어씩 제시)

5 5

문장이해 테스트 행동반응 결과

5 6 • • 어려운 문장 , 즉 목적관계절 삽입문을 읽고 대답할 때 , 반응시간이 더욱 느리고 , 더 많은 오반응을 보임 .

문장을 한 단어씩 제시했을 때 , 반응시간에서는 별다른 차이를 보이지 않았으나 , 오반응률은 조건간의 차이가 훨씬 큼 . => 문장 제시 방법이 문법적 처리 / 이해 과정에 영향을 미침 .

문장을 읽는 과정에 나타난 문장 난이도의 효과

5 7 • • •

Broca

영역에서 통사적으로 복잡한 문장을 처리할 때 보다 많은 신경세포의 활동이 관찰됨

.

그러나

,

문장을 한 단어씩 보여줬을 때는

,

문장의 난이도에 그리 상관없이

,

전반적으로 높은 신경활성화를 보임 문장처리에 관여하는 작업 기억

(Working memory)

과 문장제시방법의 상호작용

(interaction)

을 보여줌

.

문장을 읽고 난 후

,

답하는 과정에 나타나는 문장 난이도의 효과

5 8 • • • • 문장을 읽고 난 후 , 답을 하는 과정에도 읽은 문장의 난이도가 여전히 영향을 미침 .

읽은 문장의 이해정도에 따라 그 차이를 보임 Wernicke 영역에서 문장제시 방법에 따른 상호작용을 보임 한 단어씩 제시했을 경우 , 문장의 이해 정도의 차이가 없음 .

의미 표상의 문제

5 9 • • • 우리는 어떻게 의미를 표상하고 처리할까

?

지식 표상체계

-

개념

/

명제

/

도식적 지식표상 의미표상의 제 문제에 대한 인지신경과학적 접근 – – – – 구체적 사물에 대한 의미

(inferior temporal visual pathway)

행위에 대한 의미

(motor cortex)

추상적 개념에 대한 의미

(inferior parietal lobule) =>

우리가 어떻게 대상개념을 습득하고 상호작용하는가에 의해 결정되는 것 같다

.

Semantic memory in the brain Part1: objects

Neural representation for object concepts (Animal vs. Tool) adapted from (Chao & Martin, Nature, 2001) Red – Animal, Blue- Tool Semantic processing for object concepts (Martin, Annu. Rev. Psy. 2007) FFA(fusiform face area) – Kanwisher et al.(1997), McCarthy et al.(1997) PPA(parahippocampal place area)- Epstein and Kanwisher(1998) 6 0

Semantic memory in the brain Part2: actions

6 1 Neural activation for Action Words adapted from(Hauk et Somatotopic representation for action concepts adapted from (Hauk et al., Neuron, 2004)

6 2

단어의 의미에 기초한 뇌 활성화 예측

•

Predicting Human Brain Activity Associated with the Meanings of Nouns (Science, 2008)

6 3

마치며 …

• 뇌와 마음 – – – – – 뇌에 마음이

/

언어가 있는가

?

뇌는 소중한 우리 신체의 일부

,

손

/

발 과 마찬가지로

..

뇌는 우리의 정신적

/

육체적 작용의 학습과 발달로 끊임없이 변해가는 기관

.

따라서

,

정신적

/

육체적 활동의 변화를 잘 나타내는 신체기관으로

,

이를 연구함으로 인해 우리 마음의 작용을 보다 과학적으로 연구할 수 있는 토대를 제공함

.

여러분은 무엇을 연구할 것인가

?