Transcript Diapositiva 1

Ph.D. in Political Science - Comparative and European Politics
Academic year 2012-2013
Part 2 Experimental Design
EXPERIMENTAL METHODS IN POLITICAL AND SOCIAL SCIENCES
Alessandro Innocenti
[email protected]
OUTLINE
Part 1 Laboratory Methods
To provide a basic introduction to experimental methodology both from a theoretical and
an empirical point of view.
Part 2 Experimental Design
To learn how to design an experiment and to understand that experiments in
political sciences share many features from cognitive and experimental
economics.
Part 3 Applied Experiments
To understand the differences between different kinds of experimental designs by
discussing weaknesses and strengths of some experimental papers and the
specificities of their designs.
1
Part 2 Experimental Design
BIASES IN JUDGMENT
“People rely on heuristic principles which reduce the complex tasks of assessing
probabilities and predicting values to simpler judgmental operations. In general, these
heuristics are quite useful, but sometimes they lead to severe and systematic errors”
(Tversky and Kahneman 1974)
CONFIRMATION BIAS
Once individuals devise a strong hypothesis they will tend to misinterpret or even
misread new information unfavorable to this hypothesis
Also production of treatment effects: when a researcher believes an hypothesis is
true, he often produces a biased sample of evidence that reinforces his or her belief
(unconsciously?)
Consequence is obvious: confirmation bias inhibit learning whether one’s underlying
belief is false
But fresh thinkers may be better at seeing solutions to problems than people who
have meditated at length on the problems, because the fresh thinkers are not
overwhelmed by the “interference” of old hypotheses.
2
Correlated phenomena
FALSE CONSENSUS
People use their own tastes and beliefs as information in guessing what others like
and believe
Application: to put in other people’s shoes is not useful to find focal points



HINDSIGHT BIAS
Current recollections of past judgments tend to be biased by what actually happened
since then
WYSIATI RULE
what you see is all there is
We often fail to allow for the possibility that evidence that should be critical to our
judgnebt is missin (what you see is all there is)
Applications
Jumping to conclusions on the basis of limited evidence
The halo effect judgments of a person’s character can be influenced by one's overall
impression of him or her (attractiveness)
Overconfidence. The WYSIATI rule implies that neither the quantity or the quality of
the evidence counts for much in subjective confidence
3
AN ILLUSTRATIVE EXPERIMENT
Martin Jones and Robert Sugden “Positive confirmation bias in the acquisition of
information”,Theory and Decision, 50, 2001, 59-99
Positive confirmation bias: tendency, when testing an existing belief, to search for
evidence which could confirm that belief, rather than for evidence which could
disconfirm it
Wason’s (1968) selection task
Four double-sided cards. Subjects are told that each card has a letter on one side
and a number on the other, but they can see only the upper faces of the four cards
Four cards uncovered show ‘A’, ‘D’, ‘4’ and ‘7’
Each subject is asked to consider the following rule, as applied to the four cards: ‘If a
card has a vowel on one side, then it has an even number on the other side’
Instruction: ‘Your task is to say which of the cards you need to turn over to find out
whether the rule is true or false’
4
Two most common responses

‘A’ card alone

‘A’ and ‘4’ cards in combination
The correct answer to the question posed is, of course, the combination of ‘A’ and ‘7’.
The frequently-chosen ‘4’ card can provide no information which is relevant to the issue
of whether the rule is true or false
The ‘A’ and ‘4’ cards are the ones that are capable of providing evidence which confirms
the rule: by turning over either of these cards, the subject may find a card with a vowel on
one side and an even number on the other
In contrast, the ‘7’ card can only disconfirm the rule (i.e. by revealing a card which has a
vowel on one side but not an even number on the other)
In this sense, the evidence from the selection task can be interpreted as consistent with
positive confirmation bias
5
Criticism
The original selection task was formulated in highly abstract terms
Counterargument
Correct response might be facilitated by adding thematic content to the task, i.e. by
providing a cover story which accounts for the statement and gives some point to the
selection task
Jones and Sudgen’s design

Subjects have to pay a fixed cost per card turned over

After they have made this choice, the chosen cards are turned over

Then they make the judgment that the statement is ‘true’ or ‘false’

Finally the remaining cards are turned over and they receive a fixed reward if and
only if their judgment was in fact correct
6
Experiment carried out at the University of East Anglia in Norwich

120 students recruited on the campus (wide range of courses)
 Computerized experiment
 No communication between subjects
Each task is presented by means of a sequence of six screens
The screen presents first the cover story, then the statement and finally four cards to
choose
Each object has two characteristics, each of which can take one of two values that
correspond with p, p, q, and q (as before vowel and consonant, even and odd)
Each subject perform seven different tasks
<p, q> or < q, p>, if turned over, is a disconfirmation of the experimental HP
<p> <p, q> and <q, p> are confirmations
7
Exemplificative Tasks
1. Relatives . A survey is taken of 100 people in Los Angeles, Seattle, London and Norwich
who have relatives living in other cities. Each person in the survey living in Britain has
relatives in Los Angeles or Seattle and each person living in America has relatives in
Norwich or London. No one has relatives in more than one city. The details of the survey
are written down on report cards by putting the city each person lives in on one side of the
card and the city their relatives live in on the other side. A sample of four report cards is
selected. Look at whichever cards you wish to test the statement:
[Standard statement] Every person in the sample who lives in London also has a relative who
lives in Los Angeles.
[Contraposed statement] Every person in the sample who lives in Seattle also has a relative
who lives in Norwich.
2. Drinkers . Only people over the age of eighteen are allowed to drink alcohol in a pub in
Britain. A survey is carried out of 100 people in a large public house which identifies their
age and whether they are drinking alcohol or a soft drink. Each person’s details are put
down on a report card with the person’s age on one side and their drinking behaviour on
the other. A sample of four report cards is selected. To find out if the four people in the
sample are obeying the law, look at whichever cards you wish to test the statement:
[Standard statement] Every person in the sample who is drinking alcohol is also over
eighteen.
[Contraposed statement] Every person in the sample who is under eighteen is also drinking a
soft drink.
8
Results
In favour of the confirming bias hypothesis: 62% of the choices (445/720)
<No cards> 18%
<p> 14%
<p, q> 18%
Conclusions

Overwhelming evidence that subjects’ information-gathering decisions are
systematically biased in favor of information which is potentially confirming

But behavior seems to have been closer to Bayesian rationality than in many other
selection task experiments

Especially the drinkers story facilitates Bayesian rationality (why?)

What is the effect of financial incentives?

Application to economic learning: an agent who repeatedly faces the same set of
options might retain the false belief that a particular option was optimal, even after
long exposure to evidence which, rationally interpreted, would indicate the contrary
9
II. Conceptions of rationality
What do we mean by rational choice? Lots of formulations, involving assumptions of
different strength
Different forms of rationality imply different experiments to test them






Goal oriented
Satisficing behavior
Maximizing behavior
Ordinal utility maximization
Expected utility maximization
Subjective expected utility maximization
Experimental economics reveals the hidden or implicit assumption by showing
anomalies in the formulation of rationality
Consequence: there is a variety of definitions of rational individual
And what about heterogeneity?
10
Risk neutral economic man: never buys insurance, but would be willing to pay any finite
amount to participate in Petersburg paradox.
Expected utility maximizing man: buys insurance, but ignores sunk costs, and is immune
to framing effects.
Almost rational economic man (e.g. prospect theory man) has malleable reference points
and probability perceptions, but still has preferences - comfortable with non-utility Allais
choices, but doesn’t exhibit preference reversals.
Psychological man doesn’t have preferences, has mental processes. Different frames
and contexts, and different choice procedures elicit different processes - So he may
sometimes exhibit preference reversals because choosing and pricing elicit different
mental procedures.
Neurobiological man: doesn't (even) have a fixed collection of mental processes, in the
sense of psychological man. He has biological and chemical processes which influence
his behavior. Different blood chemistry leads to different mental processes; e.g.
depending on the level of lithium (or Valium or Prozac) in his blood, he makes different
decisions (on both routine matters and matters of great consequence - even life and
death). An understanding of how chemistry interacts with mental processes has proved to
be very useful, for instance in treating depression.
11
CONSTRUCTIVE REACTIONS
Expected utility
theory
 prospect theory
Exponential
discounting
Self-seeking
behavior
hyperbolic
 discounting
Equilibrium
 equilibration
ranked preferences
constructed
 preferences
Bayesian probability
judgments
 confirmation bias
 social utilities
processes of
asymmetric response to price
 increases, downward-sloping labor
supply among cab drivers
 addition and procrastination
trust and reciprocity in financial
 relationship
 drift effect, automata
information manipulation in horse
 race betting, void informational
cascades in the artistic markets
Self-fulfilling expectations in
 financial markets, focal points
12
PART 2 Experimental Design





APPLICATIONS
Trust game
Trust game (or investment game):
Two players are paired off anonymously and respectively named as the sender and
the responde
The sender is given a certain amount of money and told that he or she can keep the
entire amount or send some or all of it to the responder.
Any money passed from the sender to the responder is tripled by the experimenter
and then given to the responder.
The responder can keep the entire amount or give back some or all of it to the
sender.
When the sender receives the amount sent back by the responder the game ends.
It measure of the propensities to trust, which is the proportion of the initial
endowment sent by the sender, and to reciprocate, which is the ratio between the
amount returned and the amount received by the responder.
Backward induction solution: the responder will not send any money back.
anticipating the responder’s decision, the sender will not send any money to the
responder.
13
Results from earlier experiments are inconsistent with the conventional game theory
prediction.
Table 1. Experimental results on the trust game
Berg-DickhautMcCabe (1995)
Buchan-Croson- Schwieren
Johnson (2000)
-Sutter
(2003)
No
history
Social
history
Only
U.S.
All
Trust
51.6
53.6
49
67
Reciprocity
30.1
40.2
22
37
Burks - Carpenter Verhoogen (2003)
Chaudhuri Gangadharan
(2003)
Innocenti Pazienza
(2006)
Single
role
Both
roles, no
prior
Both
roles,
prior
Double blind
procedure
Double blind
procedure
65.7
65.0
65.2
47.3
43.3
42.2
37.6
43.6
25.9
17.1
17.5
16.2
Trust = Average fraction sent (Amount sent / Initial endowment); Reciprocity = Average fraction returned (Amount
sent back/ Amount received)
14
15
Applications
OXYTOCIN AND TRUSTING BEHAVIOR
Michael Kosfeld, Markus Heinrichs, Paul J. Zak, Urs Fischbacher & Ernst Fehr
“Oxytocin increases trust in humans“ Nature 2005
In non-human mammals, the neuropeptide oxytocin has a key role in general
behavioural regulation, particularly in positive social interactions.
Oxytocin receptors are distributed in various brain regions associated with behaviour,
including pair bonding, maternal care, sexual behaviour, and the ability to form
normal social attachments.
Thus, oxytocin seems to permit animals to facilitate approach behaviour.
HP.: oxytocin might also promote prosocial approach behaviours (such as trust) in
humans.
Recent neuroscientific finding: neuropeptides cross the blood-brain barrier after
intranasal administration
16
Double-blind study design to compare:
- trusting behaviour in a group of subjects who receive a dose of intranasal oxytocin
- trusting behaviour in a control group of subjects who receive placebo.
Experiment’s object
trust game with real monetary stakes (29+29 subjects)
Hypothesis to test:
oxytocin increases the trusting behaviour of investors  the investors in the oxytocin
group will show higher money transfers than those in the placebo group.
17
Two treatments


standard trust game
risk trust game
In B the investor faced the same choices as in the trust game but in which a random
mechanism, not the trustee's decision, determined the investor's risk.
The random mechanism in the risk experiment replicated the trustees' decisions in
the trust experiment.
Therefore, the investors faced exactly the same risk as in the trust experiment
However, their transfer decisions were not embedded in a social interaction because
there were no trustees in the risk experiment.
18
Experimental design









194 male students (mean age s.d., 22.0 3.4 yr) from different universities in Zurich
128 participants in the trust experiment and 66 subjects participated in the risk
experiment
Exclusion criteria: medical or psychiatric illness, medication, smoking, drug or alcohol
abuse
Subjects were instructed to abstain from food and drink (other than water) for 2 h
before the experiment, and from alcohol, smoking and caffeine for 24 h before the
experiment
Participants were informed at the time of recruitment that the experiment would
evaluate the effects of a hormone on decision making
16 individuals out of the original sample of 194 were excluded because of incorrect
substance administration (7 in the trust experiment, 5 in the risk experiment) or their
stated disbelief that the opponent in the trust game was actually a human being (4
participants)
Subjects received a single intranasal dose of 24 IU oxytocin (Syntocinon-Spray,
Novartis; 3 puffs per nostril, each with 4 IU oxytocin) or placebo 50 min before the
start of the experiment
Subjects were randomly assigned to the oxytocin or placebo group
In order to avoid any subjective substance effects (for example, olfactory effects)
other than those caused by oxytocin, the placebo contained all inactive ingredients
except for the neuropeptide.
19
20
21
Conclusion
oxytocin increases the investors' transfer levels in the trust experiment but not in the risk
experiment  oxytocin specifically affects trust in interpersonal interactions.
Explanations
a) oxytocin causes a general increase in prosocial inclinations
Oxytocin should affect not only the prosocial behaviour of the investors but also that of
the trustees.But trustees given oxytocin do not show more trustworthy behaviour. At
every positive transfer level (4, 8 or 12 MU), their back transfers are statistically
indistinguishable from those of placebo trustees (Mann Whitney U-tests; P > 0.243, twosided tests for each positive transfer level).
b) oxytocin does not increase the general inclination to behave prosocially. Rather,
oxytocin specifically affects the trusting behaviour of investors.
c) effect of subjects' beliefs. Oxytocin might render subjects more optimistic about the
likelihood of a good outcome. In order to address this question, we measured the
investor's subjective expectation about the trustee's back transfer after every transfer
decision. A Mann-Whitney U-test indicates that these expectations do not differ
significantly between oxytocin and placebo groups at every feasible positive transfer level
d) oxytocin helps subjects to overcome their betrayal aversion in social
interactions.
This explanation is consistent with the differing effects of oxytocin across the trust and
the risk experiments, and is further supported by the fact that investors faced a
considerable betrayal risk.
22