Transcript No Slide Title

Econ 140
Experimental Approaches
Lecture 24
Lecture 24
1
Today’s plan
Econ 140
• Experiments in economics.
• A rarity, but providing important ‘thought’ experiments for
quasi-experiments.
• How to estimate with experimental data for full and partial
compliance.
• Quasi-experiments
• Heterogeneous populations and the local average treatment
effect (LATE)
• LATE and random variation in the number of kids you may
have.
Lecture 24
2
Experiments
Econ 140
• Random controlled experiments:
– select subjects from population of interest
– random assignment to either treatment or control group
– either receive or not receive treatment
– causal effect is expected effect on outcome of interest.
• Treatment has to be assigned randomly
• Treatment is then distributed independently of any other
determinants of the outcome. Eliminated omitted variable
bias.
Lecture 24
3
Experiments (2)
Econ 140
• Estimate effect of some random assignment (e.g. training
program):
Yi = a + bXi + ui
• Where Xi is a binary variable:
– Xi = 1 if treatment
– Xi = 0 if no treatment
• Causal effect of treatment level x is difference of
conditional expectations:
E(Y|X = x) - E(Y|X = 0)
Lecture 24
4
Experiments (3)
Econ 140
• Because of random assignment: E(ui|Xi) = 0
• b^ is the difference estimator.
• Problems with experiments: two types: 1) internal
validity; 2) external validity.
• Internal validity examines the threat to inference and
understanding the causal effects for the population under
study
• External validity concerns the generalizing of the
experimental result.
Lecture 24
5
Problems with Experiments
Econ 140
• Internal Validity:
– Failure to randomize
– Failure to follow treatment
– Attrition
– Experimental (Hawthorne) effects
– Small samples
• External Validity:
– Non-representative sample
– Non-representative program or policy
– General equilibrium effects
– Treatment vs eligibility
Lecture 24
6
Regression estimators
Econ 140
• Simple estimator (Yi = a + bXi + ui) might not be the most
consistent, unbiased, or efficient.
• Differences estimator with additional regressors:
Yi = a + bXi + g1W1i + … + gjWji + ui
• If all OLS assumptions hold, then b^ is BLUE.
• Can still have a consistent estimate of b^ if u and W’s are
correlated, but providing no correlation with X. Called
conditional weak dependence.
• Estimator is:
– more efficient
– provides a check on randomization
– adjusts for conditional randomization
Lecture 24
7
Regression estimators (2)
Econ 140
• Difference in difference estimator
• Experimental data often has before and after (panel)
nature.
• Difference in difference exploits this property to allow for
differences in pre-treatment differences!
• T = treatment; C = control; difference in difference
estimator:
b^(diff-in-diff) =
(meanYT,after - meanYT,before) - (meanYC,after - meanYC,before)
also:
= DmeanYT - DmeanYC
Lecture 24
8
Regression estimators (3)
Econ 140
• As a regression: DYi = a + bXi + ui
• Where DYi (for each i before and after the treatment).
• Reasons for using this approach:
– Removes time invariant effects
– Eliminates pre-treatment differences in Y.
• Can use difference in difference with additional regressors:
DYi = a + bXi + g1W1i + … + gjWji + ui
• Same arguments apply as before.
• Useful estimation tool
Lecture 24
9
Regression estimators (4)
Econ 140
• Useful estimation tool. Has a number of possibilities:
– if multiple time period observations exist, can use panel
methods
– estimate causal effects for different groups
– estimate when there is only partial compliance
• Partial compliance: use assigned treatment level as an
instrumental variable for the actual (observed).
• Providing assigned treatment level is truly random, it can
act as an instrument.
Lecture 24
10
Regression estimators (5)
Econ 140
• Test for Random Assignment:
– X (variable indicating treatment) should be uncorrelated
with other observables. Run regression of X on W’s
and undertake F - test.
– Alternative: test initial assignment for treatment (Z) on
observables. Once again an F - test.
Lecture 24
11
Quasi-experiments
Econ 140
• Economic experiments expensive and unethical (?) (e.g.
SIME/DIME and STAR) .
• Economics has taken to using quasi experiments (also
called natural experiments).
• Quasi - experiments provide an ‘as if’ random assignment
of individuals into a treatment and control groupings. No
experiment actually took place; BUT some event, law,
policy, or rule (social, behaviorial) places some individuals
in a ‘treatment’ set and some in a ‘control’ set.
• Need random variation of some description.
• Relies heavily on the idea that the partial compliance can
be instrumented with initial assignment.
Lecture 24
12
Quasi-experiments (2)
Econ 140
• Same problems as before.
• Internal Validity:
– Failure to randomize
– Failure to follow treatment
– Attrition
– Experimental (Hawthorne) effects
– Instrument validity
• External Validity: more of a judgmental consideration: are
we examining a special case?
Lecture 24
13
Heterogeneous populations
Econ 140
• In a situation where there is unobserved variation in the
population (heterogeneous), there will be a response to
treatment for each i.
• Hence estimation becomes: Yi = a + biXi + ui.
• Note that b depends on i.
• If individuals selected at random, it follows that bi should
also be a random variable.
• When population is heterogeneous, then the causal effect
observed is the average causal effect, or local average
treatment effect (LATE).
Lecture 24
14
LATE
Econ 140
• LATE (OLS estimated) identifies only the average effect
for individuals under ‘as if’ assignment.
• LATE (IV estimated) identifies only the weighted average
of the causal effects for those who reacted greatest to the
treatment.
• In other words, the instrument identifies a weighted
average of the causal effect, where those for whom the
instrument is most influential receive the most weight.
Lecture 24
15
LATE and Angrist & Evans
Econ 140
• Understanding instrumental variables is important in
economics because of recent use of local average treatment
effects (LATE) in applied literature
• Example: Angrist and Evans AER (1998)
– tested for the effect of having children on hours worked
– looked at random instruments for a treatment and
control group
• Number of kids and propensity to have another child
– testing to see if sex of the first two affects the
likelihood of having a third
Lecture 24
16
LATE and Angrist & Evans (2)
Econ 140
• Random assignment of treatment and control groups
– can’t determine sex of children
– assumption: number of kids influences hours of work
because the more kids you have, the less time time you
have to work
• Identification comes off small part of sample: need very large
sample size
Lecture 24
17
LATE and Angrist & Evans (3)
Econ 140
• Estimated the following equation:Hi = a1 +b1Xi + b2NKi + I
– here they identifying on the number of kids (NKi)
• Why can we expect NKi to be endogenous?
– Might choose combination of number of kids and hours of
work, but can’t choose sex of children
• So Angrist and Evans used the same sex variable as the
identifying variable: NKi = g1 + g2Xi + g3samesexi + vi
– samesexi is a dummy variable equal to 1 if the children are
the same sex and 0 otherwise
– Negative result: more kids leads to less hours of work
– L24.xls has a worked example
Lecture 24
18