Transcript Experimental Finance: Responsibilities of Coming of Age Shyam Sunder, Yale University Keynote Address, Society for Experimental Finance Tilburg University Tilburg, The Netherlands, June 27-29, 2013

Experimental Finance:
Responsibilities of Coming of Age
Shyam Sunder, Yale University
Keynote Address, Society for Experimental Finance
Tilburg University
Tilburg, The Netherlands, June 27-29, 2013
As a Discipline Grows Up
• Since Chamberlin reported the results of his classroom
economics experiment in 1948, the acceptability, recognition,
role, and methods of this sub-discipline have evolved
• Similar pattern has emerged in experimental finance since the
early 1980s
• Unlike 1970s and 80s, when editors of economics, and then
finance, journals routinely rejected experimental papers as a
deviant curiosity, a recent issue of AER had more papers using
experimental method than any other
• Although its acceptability in finance lags economics, it is clear
that the experimental method has grown beyond its “childhood”
phase, is no longer “outside the tent”
• Being inside the tent brings responsibilities of “adulthood” for a
sub-discipline?
11/6/2015
Sunder: Experimental Finance
2
Responsibilities of a Mature Discipline
• Identifying core concerns of the discipline on substantive,
not just methodological grounds;
• Going beyond show-and-tell to impress “adults”. No more:
– Look no Hands, Ma!
– I can do what you can do.
• Contribution to core concerns of a discipline
• Constructive interchange with sister methodologies of the
discipline
• Balance between advancement of method and substantive
knowledge of real phenomena
• Statistical methods permit rejection of null or failure to
reject the null
– No such thing as confirmation of hypothesis
– The null is the current
belief;
not arbitrary
11/6/2015
Sunder:
Experimental
Finance choice
3
Seven Special Concerns
•
•
•
•
•
Robustness
Time scale
Risk: what is it?
Institutions
Properties of institutions, or individual behavior
– Individual behavior: distinguish unobservable traits from
observable actions (vs. Re-labeling actions as traits)
• Is the experimenter a part of the game; subject expectations
• Laboratory results as the final word, and the main source
of research questions
11/6/2015
Sunder: Experimental Finance
4
Core concerns of the discipline:
substantive, not just methodological
• Disciplines get sterile when methods take the front seat,
obscuring their classic or newly-identified substantive questions
• While methodological development is necessary part of a
healthy discipline, the dominant concern must still be with a
better understanding of relevant aspects of the world we live in
• What proportion of the effort of the discipline goes into research
about questions about the world (external references), and
questions about research itself and its methods (internal
references)?
• A simple test: try explaining our research question (and results)
to our parents, (any non-expert, really) to assess if they
appreciate what we contribute to human civilization with the
resources we get to spend
11/6/2015
Sunder: Experimental Finance
5
Contribution to core concerns of a discipline
• Where do we look for questions to address?
– On the street, news, and direct observation of the world
– Questions arising in the classroom (by students as well as in our
own minds) that we cannot answer to their or our satisfaction
– Unresolved (perhaps abandoned) puzzles of the discipline
– Incremental variations on recent publications
– Proving your advisor or academic god-parent right
– Because it may get published
• Identifying core concerns of economics/finance that could not
be addressed without experiments
• What are the core concerns of economics/finance and sister
disciplines such as psychology?
• Do we need to distinguish among them? Does making
distinction imply un/willingness to learn from others?
11/6/2015
Sunder: Experimental Finance
6
Constructive interchange with
sister methodologies
• Contribution of experimental method will also
depend on how well we are able to take
advantage of constructive interchange with
sister methodologies of economics/finance
• For example, theory and mathematical
modeling; econometric modeling and
estimation/testing with data from the field
• What can be a constructive relationship
between theory and experiments?
11/6/2015
Sunder: Experimental Finance
7
Robustness and Assumptions
• We build models is to gain a better understanding of some
real phenomena of interest
• Models may use mathematics to understand the world, but
are not mathematics (do not pursue math for its own sake)
• Real phenomena are complex (perhaps infinitely detailed);
rarely possible to understand/characterize them completely
• Theory identifies one, or a few, critical variables from a
large set to gain a satisfactory (not perfect) understanding
of the phenomenon of substantive interest
• Theories are neither wrong nor right; some are more
helpful/robust than others in explaining/predicting/gaining
insight
• Compare theories on basis of their helpfulness in
explaining/understanding the real phenomena of interest
11/6/2015
Sunder: Experimental Finance
8
Fractals: Infinite Detail
11/6/2015
Sunder: Experimental Finance
9
Nature of Theory
• Essence of theory lies in its simplicity; we understand by
simplifying
• Simplification by abstraction from details of real
phenomena
• Assumptions are the way of discarding the great mass of
detail to focus on one/few key factors; and adding a few
for analytical convenience
• Every model consists of : key assumptions and
assumptions of convenience
• Lack of correspondence between assumptions of
convenience and reality is the essence of a theory, and not
a defect of theory (no assumptions, no theory)
• Key assumptions must hold; convenience assumptions
need not
11/6/2015
Sunder: Experimental Finance
10
Empirical Test of a Theory
• Theory is to real phenomena what a drawing or stick
figure is to the human body, or a map is to earth’s
surface
• Correspondence is crude on purpose; it captures some,
and only some, essential feature(s) which are relevant
to the purpose on hand
• Model identifies some tautologies which are
necessarily true when assumptions hold (unless there
are mathematical errors in derivation):
If (x, y, z)  P
• What does it mean to empirically test a theory in the
laboratory?
11/6/2015
Sunder: Experimental Finance
11
Single Theory Experiments
• When only one interesting theory is available for
the phenomenon of interest
• “Test” is an assessment of robustness of the
theory to deviations from assumptions of
convenience
• If data are gathered from an environment that
corresponds exactly to the assumptions of the
theory, we should expect no deviations (if we do
observe deviations, either the theory has error or
the correspondence is missing)
• Empirical test is a costly method of discovering
errors of derivation
11/6/2015
Sunder: Experimental Finance
12
Creating Theoretical Model in Lab
• Exact correspondence to theory in either the field
or lab is rarely achievable
• Even if we could, little can be learned from it
except about any math errors of derivation or in
lab/model correspondence; expensive way to
discover errors
• Error in derivation or lab environment or data
collection
• Little useful scientific inference is possible from
perfect correspondence between the theoretical
model and its laboratory implementation
11/6/2015
Sunder: Experimental Finance
13
Scientific Value of A Single-theory
Empirical Test
• Assessment of how robustly the predictions of the theory
correspond to data as more of the convenience
assumptions are relaxed in the environment of data source
• See Figure 1:
• At the origin, x-axis shows that data is gathered from a lab
environment in which all assumptions of the model (core
as well as convenience) hold; i.e., zero distance between
them
• Under these conditions the correspondence between the
data and the model prediction should be perfect.
• If it is not found to be perfect, what could the reasons be?
– Mathematical error in the model
– Lack of correspondence between the model and the lab
environment, i.e., error in implementing the model in lab
11/6/2015
Sunder: Experimental Finance
14
Single Theory Experiment
100%
Corresponde
nce between
the Data and
Theory
Prediction
0%
0
Distance between the model and
the data environment
11/6/2015
Sunder: Experimental Finance
15
What Can We Learn from Such a
Lab Experiment?
• Not much
• If there is a math (derivation) error in the model, laboratory
experiment is an expensive way for finding out such errors
(try Mathematica instead!)
• If the lab environment does not exactly replicate all (core
and convenience) assumptions of the model, there is error
of implementation.
• Again, actually running the experiment is an expensive
way of finding out such errors
• In neither case, does the experiment enlighten us with an
answer to the inquiry
11/6/2015
Sunder: Experimental Finance
16
What Can Experiments Do?
• Systemcatically relax/drop the assumptions of
convenience in the model
• Remove these assumptions one by one, and move the
design of the experiment to the right on the x-axis
• For each design, assess the degree of correspondence
between the model prediction and the data
• See how rapidly the explanatory power of the model
declines as the lab environment drop more
assumptions of convenience
• See A, B and C as three candidate models, for
example
11/6/2015
Sunder: Experimental Finance
17
Single Theory Experiment
100%
A
Corresponde
nce between
the Data and
Theory
Prediction
0%
0
Distance between the model and
the data environment
11/6/2015
Sunder: Experimental Finance
18
Single Theory Experiment
100%
A
Corresponde
nce between
the Data and
Theory
Prediction
B
0%
0
Distance between the model and
the data environment
11/6/2015
Sunder: Experimental Finance
19
Single Theory Experiment
100%
A
Corresponde
nce between
the Data and
Theory
Prediction
B
C
0%
0
Distance between the model and
the data environment
11/6/2015
Sunder: Experimental Finance
20
Robustness of Predictive Power
to Assumptions of Convenience
• Under all three cases, the model is literally true (when
all its assumptions hold).
• However, as the environment deviates from the strict
assumptions, A’s predictive power declines more
slowly than B’s
• C’s predictive power declines very rapidly
• C is valid literally, but its power to explain the world
we live in is likely to be limited, compared to A and B
• Which would you favor as a better model/theory?
11/6/2015
Sunder: Experimental Finance
23
How Do We Identify Key the
Assumptions?
• Distinguishing between model and theory
• Model is a (“stick figure”) logical structure; a theory uses
the model to suggest some statements about the real
phenomena of interest
• Think of the real phenomena that motivates the model and
the theory as the principal
• Ask which assumptions of the model are intended to limit
the real environments sought to be understood; those are
the key assumptions
• Number of states, preferences, probability distributions,
etc., tend to be assumptions of convenience, because they
are not intended to limit application of the model (unless
you are interested only in a 2-state world, for example)
11/6/2015
Sunder: Experimental Finance
24
Design of Robustness
Experiments
• Conduct a series of experiments, all holding the key
assumptions, and progressively relaxing the
convenience assumptions (e.g., the number of states)
• Conduct a series of experiments progressively
increasing the number of alternative choices available
to subjects (increasing the number of possible
outcomes)
• If the predictive power of the model is relatively
robust as shown by data when more alternatives are
available, result is more robust; e.g.,Vernon Smith
(1962) paper
11/6/2015
Sunder: Experimental Finance
25
Fig. 3: Single Theory Experiment
Smith (1962)
11/6/2015
Sunder: Experimental Finance
26
Why Was the Smith (1962) Experiment
So Powerful?
• It dropped a whole basketful of assumptions
which had been thought to be at the core, but
turned out to be mere convenience in the basic
supply-demand equilibrium model
– No tatonnement
– Hardly perfect competition
– Only private erfect information
– Profit motivated, but hardly optimizers
• Showed the model to be far more robust than even
the most ardent supporters had claimed (or even
imagined)
11/6/2015
Sunder: Experimental Finance
27
But There Was More!
• Gode and Sunder (JPE
1993): Combine
– Double auction market
institution (without
randomization in Smith),
and
– Budget constrained random
choice (without market
institution in Becker 1962)
– Found that allocative
efficiency is largely a
function of the double
auction (ZI traders)
11/6/2015
Sunder: Experimental Finance
28
Market Behavior 1
11/6/2015
Sunder: Experimental Finance
29
Market Behavior 3
11/6/2015
Sunder: Experimental Finance
30
Figure 2: Multi-Theory Experiments
Predictions
Results
11/6/2015
W
Theory 1
Theory 3
Theory 2
X
Y
Sunder: Experimental Finance
Z
31
Fig. 4: Multi-Theory Experiment,
Plott and Sunder (JPE 1982)
11/6/2015
Sunder: Experimental Finance
32
But How and Why?
• While Plott and Sunder (1982) found that double
auction markets could disseminate information
from insiders to non-insiders with human traders
• Human faculties are sufficient, but are they
necessary?
• Again, Jamal, Maier and Sunder (2012) replaced
human traders in Plott and Sunder market design
by minimally intelligent traders to find out if
double auction markets without the entire portfolio
of human faculties can still achieve this task
11/6/2015
Sunder: Experimental Finance
33
Design of Algorithmic Traders
• Populate double auctions with simple
algorithmic traders with two characteristics
– Means-end heuristic to adjust their aspiration
levels
– Zero-intelligence bids and offers relative to the
current aspiration levels
11/6/2015
Sunder: Experimental Finance
34
Means-End Heuristic
• Use Newell and Simon (The Simulation of Human
Thought, 1959)
– Given a current state and a goal state, an action is chosen which
will reduce the difference between the two. The action is
performed on the current state to produce a new state, and the
process is recursively applied to this new state and the goal state
• Initial aspiration
– For the uninformed: expected payoff
– For the informed: pay off under the known state
• Adjustment of aspiration levels with each observed price
P(t) using parameter α ~U(0.05, 0.5)
– ASP (t+1) = (1-α) ASP (t) + α P(t)
11/6/2015
Sunder: Experimental Finance
35
Zero-Intelligence Bids and
Offers
• Bids:
• Offers:
~ U(0, ASP)
~U(ASP, 1)
• Double Auction Rule
– At the start of each period
– Set current bid = 0, and current ask = 1
– A higher bid replaces current bid, and a lower ask
replaces current ask
– As soon as a current bid equals or exceeds current ask,
a transaction is recorded at the bid/ask submitted first
11/6/2015
Sunder: Experimental Finance
36
Results
• Plott and Sunder (1982) results of markets with
profit-motivated human traders shown in blue
• Simulated results of same markets populated by
minimally-intelligent algorithmic traders shown in
red (cloud of 50 independent simulations and the
median, α ~U(0.05, 0.5) drawn for each simulation)
• RE equilibrium price in green line
• Walrasian equilibrium price in broken line
• Also shown, comparisons of mean squared
deviation, trading volume, and allocative efficiency
11/6/2015
Sunder: Experimental Finance
37
11/6/2015
Sunder: Experimental Finance
39
11/6/2015
Sunder: Experimental Finance
40
11/6/2015
Sunder: Experimental Finance
41
11/6/2015
Sunder: Experimental Finance
42
11/6/2015
Sunder: Experimental Finance
43
11/6/2015
Sunder: Experimental Finance
44
11/6/2015
Sunder: Experimental Finance
45
11/6/2015
Sunder: Experimental Finance
46
11/6/2015
Sunder: Experimental Finance
47
11/6/2015
Sunder: Experimental Finance
48
11/6/2015
Sunder: Experimental Finance
49
11/6/2015
Sunder: Experimental Finance
50
11/6/2015
Sunder: Experimental Finance
51
11/6/2015
Sunder: Experimental Finance
52
11/6/2015
Sunder: Experimental Finance
53
11/6/2015
Sunder: Experimental Finance
54
1. Robustness Summary
• Experimental method is a powerful device to
subject economics/finance models and theories to
robustness check to help distinguish those which
hold a greater promise to give us a better
understanding of the world we live and work in
• Checking robustness of a model calls for
observations in lab environment that is designed to
deviate in assumptions of convenience from the
model(s) under scrutiny
11/6/2015
Sunder: Experimental Finance
55
2. Time Scale
• Most economic models include time dimension (usually
denoted by symbol t
• Few models specify what t represents in real terms—
seconds, hours, days, years, or generations
• Presumably, such theories are so general that they holds
for all interpretations of the time interval in real units
• Lab experiments could be a way of finding the appropriate
interpretations of time in specific theories, in case they
exist, and thus make a significant contribution of
economic/finance theory
11/6/2015
Sunder: Experimental Finance
56
3. Risk: What Is It?
• Dispersion or possibility of loss?
• What is the empirical content of risk attitudes (expected utility)
applied to curved functions of a increasingly rococo variety
• Predictive content out of sample and context
• Evidence on macro phenomena (medicine, sports, drugs,
gambling, insurance, credit, equity, labor, monetary, real estate
parts of economy
• Is risk attitude a scientific concept, a modern day equivalent of
the eighteenth century “phlogiston” in chemistry to explain
combustion, or just a plug for whatever we do not know?
• What has been the contribution of experimental finance to this
key concept of finance theory and practice?
11/6/2015
Sunder: Experimental Finance
57
4. Institutions
• Experimental methods have highlighted the importance of
economic institutions, their properties, and their evolution
over time
• However, study of institutions in lab presents a special
challenge
• Most individual decisions involve choice of a point on a
function, but institutions being functions themselves,
examination of their evolution calls for choices from a set
of functions
• Choice on a function and of a function call for very
different cognitive skills, experience, and time, and are
difficult to study in the few hours of a typical session
11/6/2015
Sunder: Experimental Finance
58
5. Properties of Institutions
(or Individual Behavior)
• Experiments have been employed to identify the properties
of institutions
• Real life institutions have great deal of detail, and thus can
be simplified for laboratory use in thousands of ways
• When we try to use experiments to identify institutional
properties, how do we choose which implementation of the
institution in the lab is appropriate?
• I have found no convincing answer to guide me in deciding
on the lab implementation of a real life institution other
than dropping assumptions of convenience mentioned
above
11/6/2015
Sunder: Experimental Finance
59
6. Experimenter as a part of the
game
• What are the boundaries of the game we
hypothesize the subjects to be playing?
• What do we know about the expectations subjects
bring to the lab? What, if any, control can we
exercise on their expectations
• Is experimenter inside that boundary or outside?
• How do we keep ourselves outside the boundary?
• Is it enough to tell them so?
11/6/2015
Sunder: Experimental Finance
60
7. Lab results as the final word?
• When can we stop with the lab results,
convinced that we have a good
understanding of the phenomenon of
interest?
• When do we need to follow up the lab
results with data from the field?
• More engagement with sister research
methods
11/6/2015
Sunder: Experimental Finance
61
Fundamental Principle of Research
Designs (after Einstein)
• Research design should be as simple as
possible, but no simpler, to answer the
question posed.
11/6/2015
Sunder: Experimental Finance
62
Research Questions
• What question do you wish to answer with your research?
• A question is one sentence with a question mark at the end
(?).
• It should be a question whose answer you would like to
know, but do not know
• After asking your friends, if you are the only one who does
not know, think again, unless you have reasons to disagree
with them
• What might the possible answers to the question?
• How could one distinguish what is a better answer?
• What is the best way to answer the question? Not
necessarily an experiment
11/6/2015
Sunder: Experimental Finance
63
Finance Challenges for
Experimental Method
• Risk
• Information and individual decision making
• Interplay between financial decisions,
reporting, and engineering
• Financial institutions
• Financial regulation and laws
• Robustness check
• Finance is related to, but is not economics
11/6/2015
Sunder: Experimental Finance
64
[email protected]
www.som.yale.edu/faculty/sunder/res
earch
11/6/2015
Sunder: Experimental Finance
65