Transcript Diapositiva 1

Ph.D. in Political Science - Comparative and European Politics
Academic year 2012-2013
Part 1 Laboratory Methods
EXPERIMENTAL METHODS IN POLITICAL AND SOCIAL SCIENCES
Alessandro Innocenti
[email protected]
Outline
Part 1 Laboratory Methods
To provide a basic introduction to experimental methodology both from a
theoretical and an empirical point of view.
Part 2 Experimental Design
To learn how to design an experiment and to understand that experiments in political
sciences share many features from cognitive and experimental economics.
Part 3 Applied Experiments
To understand the differences between different kinds of experimental designs by
discussing weaknesses and strengths of some experimental papers and the
specificities of their designs.
1
Course Programme
Class Laboratory Experiment
Experimental Gaming, Solution Concepts and Cognitive Biases
Reference to very elementary concepts of game theory is required
(see http://www.labsi.org/innocenti/introgt.pdf)
Part 1 Laboratory Methods
Outline: Purposes of Experiments (Why?). Experimental Methods (how?). Professional
Subjects, Students or What? Experimental Topics. Internal and external validity.
Virtual experiments. A laboratory experiment will be performed at the start of the
course and discussed during the class
Literature:
*Davis, D.D. and C.A. Holt (1993) Experimental Economics, Princeton University Press,
Princeton, Chapt. 1-2.
*Friedman, D. and S. Sunder (1994) Experimental methods. A primer for economists,
Cambridge University Press, Cambridge, Chapt. 1-2
*Friedman, D and A. Cassar (2004) Economics Lab. An intensive course in experimental
economics, Routledge, London and New York, Chapt. 2-3
*Smith, V. (1994) “Economics in the Laboratory”, Journal of Economic Perspectives, 8,
113-131.
2
Part 2 Experimental Design
Outline: Two illustrative experiments. Biases in judgment: the case of confirmation bias.
Trust game. Oxytocin and trusting behavior.
Literature:
*Jones, M. and R. Sugden (2001) “Positive confirmation bias in the acquisition of
information”, Theory and Decision, 50, 59-99.
*Kosfeld, M., M. Heinrichs, P. J. Zak, U. Fischbacher and E. Fehr (2005) “Oxytocin
increases trust in humans“, Nature, 435, 673-676.
*Morton R.B and K.C. Williams (2010) Experimental Political Science and the Study of
Causality. From Nature to the Lab, Cambridge University Press, New York, Ch. 13.
Part 3 Applied Experiments
Outline: Informational cascades and overconfident behavior. Voting by ballots and by
feet. Travel mode choice and transportation policy.
Literature:
*Innocenti, A., A. Rufa and J. Semmoloni (2010) "Overconfident behavior in
informational cascades: An eye-tracking study", Journal of Neuroscience,
Psychology, and Economics, 3, 74-82.
*Innocenti A. and C. Rapallini (2011) "Voting by Ballots and Feet in the Laboratory",
Giornale degli Economisti e Annali di Economia, 70, 3-24.
*Innocenti, A., P. Lattarulo and M.G. Pazienza (2013) "Car Stickiness: Heuristics and
Biases in Travel Choice", Transport Policy, 25, 153-168.
3
Part 1 Laboratory methods
POINTS OF VIEW
PROS
“Would it not be better to leave laboratory experiments to psychologists who are
trained to run them properly? Nobody doubts that we have a great deal to learn from
psychologists about laboratory technique and learning theory, but recent history
would nevertheless suggest that the answer is a resounding no. Our comparative
advantage as economists is that we not only understand the formal statements of
economic theory, but we are also sensitive to the economic environments and
institutions within which the assumptions from which such statements are deduced
are likely to be valid. Just as chemists know not to mix reagents in dirty test tubes, so
we know that there is no point in testing economic propositions in circumstances to
which they should not reasonably be expected to apply.”
(Binmore 1999)
“Once models, as opposed to economies, became the focus of research the
simplicity of an experiment and perhaps even the absence of features of more
complicated economies became an asset. The experiment should be judged by the
lessons it teaches about theory and not by its similarity with what nature might
happen to have created.”
(Plott 1991)
4
POINTS OF VIEW
CONS
The laboratory is not a socially neutral context, but is itself an institution with its own
formal or informal, explicit or tacit, rules
Human agency takes place within a socio-economic world that is structured in the
sense that it consists of internally-related positions and systems
Experimentation in economics is likely to be of limited value, save for situations –
such as auctions – that exist in conditions of relative isolation and are characterized
by low internal complexity
(Siakantaris 2000)




experimental situations often project a game-like atmosphere in which a ‘subject’
may see himself as ‘matching wits’ against the experimenter
experimental subjects are cast in roles and they can act in accordance with his
(mis)perceptions of these roles
experiments have too short horizons (real world lasts many years and many trials)
human beings are capable to control their behavior through the implementation of
abstract rules
(Cross 1994)
5
DATA SOURCES
Happenstance
(uncontrolled conditions ongoing processes)
Experimental
(controlled conditions deliberately created)
Field
(naturally occurring
environment)
National Accounts
Commodity Prices
Income Maintenance Experiments
Field Experiments
Laboratory
(artificial environment)
Casual Processes in the Lab
Discovery of Penicillin
Choice Experiments
Auctions Simulation
Laboratory Asset Markets
HOW?
WHERE?
EXPERIMENTAL ECONOMICS
LABORATORY
+
EXPERIMENTS
(artificial environment)
+ (controlled ad hoc conditions)
6
PURPOSES OF EXPERIMENTS (WHY?)
1) Test of Behavioral Hypotheses.
by constructing a laboratory environment that satisfies as many of the structural
assumptions of a particular theory, it is possible to verify its behavioral implications
2) Theory Stress Tests
to examine the sensitivity of a theory to violations of obviously unrealistic
assumptions
3) Searching for Empirical Regularities
heuristic experiments to discover and document stylized facts
(Davis-Holt, Experimental Economics 1994)
a) Speaking to Theorists
b) Searching for Facts
c) Whispering in the Ears of Princes
(Roth 1986)
7
EXPERIMENTAL METHODOLOGY (HOW?)
1. PROCEDURAL REGULARITY
to permit replications that the researcher and observers would accept as being
valid
- instructions
- subject pool and methods of recruiting subjects
- experimental physical environment
- computerized or manual
2. MOTIVATION
- Induced-value theory: use of a reward medium allows to induce pre-specified
characteristics in experimental subjects and to make subjects’ innate
characteristics largely irrelevant
- monotonicity: subjects prefer more reward medium to less and not become
satiated
- salience: rewards are explicitly and unambiguously connected to the decisions
made
- dominance: changes in subjects’ utility from the experiment come mainly from the
reward medium and other subjective costs or benefits are rendered negligible
by comparison, i.e. others’ reward
8
3. UNBIASEDNESS
Experiments should be conducted in a manner that does not lead participants to
perceive any particular behavioral pattern as being correct or expected, unless
explicit suggestion is a treatment variable - double blind setting
4. CALIBRATION
The design has to pre-specify and to cleanly separate the experimental predictions of
alternative theories.
5. DESIGN PARALLELISM
Results established in the lab hold in other, especially non-lab, real-world situations
where similar ceteris paribus conditions hold
Vernon Smith’s parallelism precept (1982): “Propositions about the behavior of individuals
and the performance of institutions that have been tested in laboratory microeconomics
apply also to non-laboratory micro economies where similar ceteris paribus conditions
hold”
Charles Plott (1982): “While laboratory processes are simple in comparison to naturally
occurring processes, they are real processes in the sense that real people participate for
real and substantial profits and follow real rules in doing so. It is precisely because they
are real they are interesting”
9
PROFESSIONAL SUBJECTS, STUDENTS or WHAT?
Main Subjects pool - Undergraduate students




readily accessible
low opportunity costs
steep learning curve
they do not know much about experimenter’s hypothesis
PhD students
unreliable subjects because they get interested in what are you doing and respond to
their understanding of your topic rather than to incentives you have constructed
Classes or friends
dominance or salience at risk, conflicts between personal, teaching and scientific
aims
10
Professional subjects
 comparisons show that students are more adept at maximizing their profits and
learning in the lab
 high opportunity costs
 pre-specified and innate characteristics are too strong
 when involved in laboratory markets they attempt to apply rules of thumb, which,
valuable for dealing with uncertainty in the parallel natural market, are meaningless
guides in the lab.
Controversial evidence
Burns (1985): professional wool buyers and students in a progressive auction
(professionals apply familiar rules and not adjust to design requirements)
Dyer, Kagel, and Levin (1985): bidding behavior of students and construction workers
(no difference)
Dejong et al (1988): Businessmen and students in sealed-offer markets (same profits,
but higher variance for businessmen)
What about gender, age, risk attitude, experience?
11
12
Kagel, John H. - Roth, Alvin E.
The Handbook of Experimental Economics
INDEX
a) public goods
cooperation vs. selfishness (social dilemmas, free-riding, institutions)
what improves cooperation (thresholds, learning)
b) coordination problems
experiments with overlapping generations
coordination games with Pareto ranked equilibria
decentralized matching environments
c) bargaining experiments
agreements
causes of disagreements and costly delays
bargaining protocol and preplay communications
13
d) industrial organization
trading institutions centralized and decentralized
monopoly regulation and potential entry
market structure and market power
collusion factors
product differentiation and multiple markets
e) experimental asset markets
informational efficiency of markets
state-contingent claims and bubbles
learning and dynamics of adjustment paths
investment and public policy
f) auctions
symmetric independent private-values models
common value auctions
collusion
g) individual choice behavior
14
INDIVIDUAL CHOICE BEHAVIOR
I.
JUDGMENT
A. Calibration
1. Scoring Rules
2. Confidence Intervals
B. Perception and Memory Biases
C. Bayesian Updating and Representativeness
1. Underweighting on Likelihood Information (Conservatism)
2. The Law of Small Numbers and Misperceptions of Randomness
D. Confirmation Bias and Obstacles to Learning
E. Expectations Formation
F. Iterated Expectations and the Curse of Knowledge
1. False Consensus and Hindsight Bias
2. Curse of Knowledge
G. The Illusion of Control
15
II. CHOICE UNDER RISK AND UNCERTAINTY
A. Mounting Evidence of Expected Utility Violation (1965-1986)
1. The Allais Paradoxes
2. Process Violations
3. Prospect Theory
4. Elicitation Biases
B. Generalizations of Expected Utility and Recent Tests
1. Predictions of Generalized EU Theories
2. Empirical Studies Using Pair-wise Choices
3. Empirical Studies Measuring Indifference Curves
4. Empirical Studies Fitting Functions to Individuals
5. Cross-Species Robustness: Experiments with Animals
C. Subjective Expected Utility
1. The Ellsberg Paradox
2. Conceptions of Ambiguity
D. Choice over Time
16
II. CHOICE UNDER RISK AND UNCERTAINTY / II
E. Description Invariance
1. Framing Effects
2. Lottery Correlation, Regret, and Display Effects
3. Compound Lottery Reduction
F. Procedure Invariance
1. New Evidence of Preference Reversal
2. Arbitrage and Incentives
3. Reversals and Markets
4. Social Comparisons and Reversals
G. Endowment Effects and Buying-Selling Price Gaps
1. Market Experiments
2. Endowment Effects: Some Psychology and Implications
K. Search
1. Search for Wages and Prices
2. Search for Information
17
Many experimental economists seem to view their
enterprise as akin to silicon chip production.
Subjects are removed from all familiar contextual
cues. Like the characters 'thing one' and 'thing two'
in Dr. Suess' Cat in the Hat, buyers and sellers
become 'persons A and B', and all other
information that might make the situation familiar
and provide a clue about how to behave is
removed.
George Loewenstein (1999)




The context-free experiment is an elusive goal and
not necessarily a good thing
Games in the laboratory are usually played without
labels but subjects inevitably apply their own labels
A major discovery of cognitive psychology is how
all forms of thinking and problem solving are
context-dependent (language comprehension)
The laboratory is not a socially neutral context, but
is itself an institution with its own formal or
informal, explicit or tacit, rules


Internal validity - ability to draw confident causal
conclusions from one's research
External validity - ability to generalise from the
research context to the settings that the research is
intended to approximate

Experiments have the reputation of being high in
internal validity but low in external validity

Field studies of being low in internal validity but
high in external validity

One of the basic tenets of laboratory methodology is that the use
of non-professional subjects and monetary incentives allows
making subjects’ innate characteristics largely irrelevant

In some experiments, it is as if subjects take into the lab the
preferences applied to real choices and stick to them with high
probability

These biases or inclinations tend to override the incentives effect

Labels may give subjects clues to become less and not more
rational

Subjects’ behavior depends more on prior learning outside the laboratory than on
expected gains in the laboratory

Labels have the power to increase external validity with a minimal sacrifice of the
internal validity

To test learning and cognitive models, it is necessary to remind and to evoke
contexts which may activate emotions, association, similarities in the laboratory

The use of presentations with virtual reality (VR) visualisations can convey objectively
this kind of information

A Virtual Experiment combines insights from virtual reality (VR) simulations in
computer science, naturalistic decision making (NDM) and ecological rationality from
psychology, and field and lab experiments from economics


The methodological objective of Virtual
Experiments is to combine the strengths of the
artificial controls of laboratory experiments with
the naturalistic domain of field experiments or
direct field studies
In a virtual experiment the internal validity of
controlled lab experiments is joined with the
external validity of field experiments



Fiore et al. 2009
Virtual Experiment to elicit subjective risk perception
from wild fires and the opportunity cost of public
funds allocated to prescribed burns
Subjects experience four dynamic visual simulations of
specific wild fires, with varying weather and fuel
conditions. Simulations are selected to represent high
and low risk of fire damage
Participants experience a sense of presence, a
psychological state of “being there and take decisions
closer to real behavior (with cognitive constraints )
ALBO PROJECT
The ALBO project is funded by the Tuscany Region in the framework
of PAR FAS 2007-2013 Linea di azione 1.1.a.3 in agreement with the
University of Siena.
http://www.progettoalbo.it/index.php?lang=en
to elicit factor of risks and job related stress in the workplace

by making interactive the simulation of “L’enterprise virtuelle” by
Government of France
http://www.travailler-mieux.gouv.fr/