Transcript Low-income children’s self-regulation in the classroom

Fadeoutin in
human
capital
Fadeout
human
capital
interventions:
interventions:
Death,Death,
miracles
and
resurrection
miracles
and resurrection
1.00
0.75
0.50
IQ
0.25
Greg J. Duncan
Drew Bailey
Winnie Yu
Earnings
School of Education
University of California, Irvine
0.00
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Age
-0.25
Impact
Death
1
Program period
0.75
0.5
0.25
0
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22
-0.25
Age
Solid marker denotes p<.05
0.30
0.20
0.10
End of program
Program period
Effect size in sd units
0.40
Letter-Word
Math
0.00
-0.10
3
4
Solid marker denotes p<.05
K
1st
3rd
Effect size in sd units
0.40
End of program
0.30
0.23
0.20
0.10 0.09
0.10
0.05
0.06
2-4
4+
0.00
0
0-1
1-2
Solid marker denotes p<.05
Percentage Employed Full-Time
30.0
20.0
Program period
10.0
0.0
0 3 7 11 15 19 23 27 31 35 39 43 47 51
-10.0
Months From Random Assignment
Impact
Persistence
1.25
Abecedarian
1
Perry
0.75
Abecedarian
0.5
0.25
Perry
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
-0.25
Age
1
0.75
Age 27
earnings
IQ
0.5
0.25
0
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
-0.25
Age
Solid marker denotes p<.05
$2,500
Adult women
Adult men
Female youth
$2,000
$1,500
$1,000
Program
period
$500
$0
7-18
-$500
19-30
Months since random assignment
Solid marker denotes p<.05
Patterns of fade-out
A mess!
• Impacts fade out in some interventions but
don’t in seemingly similar interventions
• Sometimes, for the same intervention, some
impacts fade out but others emerge decades
later
Outline
I. OPTIMAL CONDITIONS FOR IMPACT
PERSISTENCE
II. OTHER AVENUES FOR IMPACT
PERSISTENCE
III. HOW TO RECONCILE ECE PROGRAM
FADEOUT ON IQ WITH IMPACTS ON ADULT
OUTCOMES?
Outline
I. OPTIMAL CONDITIONS FOR IMPACT
PERSISTENCE
II. OTHER AVENUES FOR IMPACT
PERSISTENCE
III. HOW TO RECONCILE ECE PROGRAM
FADEOUT ON IQ WITH IMPACTS ON ADULT
OUTCOMES?
What conditions lead to impact
persistence?
When interventions change:
• the “right kinds” of skills, or
• the “right kinds” of environments
What conditions lead to impact
persistence?
When interventions change:
• the “right kinds” of skills, or
• the “right kinds” of environments
A note on “skills”
The term “skill” will be used broadly to encompass:
• Skills, behaviors, capacities or psychological
resources that help individuals attain successful
outcomes:
– IQ, knowing fractions, impulse control, positive selfconcept
“Right kinds” of skills for impact
persistence:
• Skills fundamental for success in
adulthood or for childhood attainments…
• that are malleable through intervention…
• and would not develop eventually in
most counterfactual conditions
“Right kinds” of environments
for impact persistence:
• Malleable features of environments
that are fundamental for
promoting the “right kinds” of skills
and behaviors
Fundamental and malleable skills
Fundamental
More
malleable
?...
• Conscientiousness
(grit)
Less
malleable
• g (IQ)
Peripheral
• Teaching to the test
SAT test prep
Flash cards
• FAFSA rule knowledge
Who cares?
Fundamental AND malleable
skills?
• Math
• Literacy
•
•
•
•
•
•
• Fixed vs. malleable
intelligence (Dweck)
Implicit theories (Dweck) • Self-worth (Cohen et al.)
• Math: number line,
Self-concept (Cohen)
fractions, algebra
Academic motivation
• Literacy
Executive function
• Background knowledge
Emotional self-regulation
• Executive function
Background knowledge
• Prosocial behaviors
“Right kinds” of skills:
fundamental for success…
malleable…
and would not develop eventually in
most counterfactual conditions
Fundamental AND malleable
skills?
• Math
• Literacy
•
•
•
•
•
•
• Fixed vs. malleable
Which would
intelligence
(Dweck)
develop
eventually
•
Self-worth
(Cohen et al.)
Implicit theories (Dweck)
and
therefore
•
Math:
number
line,
Self-concept (Cohen)
fractions,
algebra
generate
impact
Academic motivation
• Literacy
fadeout?
Executive function
• Background knowledge
Emotional self-regulation
• Executive function
Background knowledge
• Prosocial behaviors
Pace of development in
counterfactual conditions
Null/slow => persistence?
Eventually => fadeout?
• Math
• Literacy
• Implicit theories (Dweck)
• Self-concept (Cohen)
• Academic motivation
• Executive function
• Emotional self-regulation
• Background knowledge
Pace of development in
counterfactual conditions
Null/slow => persistence?
Eventually => fadeout?
• Fractions, algebra
• Large vocabulary
• Counting
• Letters, word sounds
• Implicit theories (Dweck)
• Self-concept (Cohen)
• Academic motivation
• Implicit theories (Dweck)
• Self-concept (Cohen)
• Academic motivation
• EF working memory
• EF impulse control
• Emotional self-reg for some • Emotional self-reg for most
• Background knowledge
• Background knowledge
Examples of impact persistence
• Algebra
– Chicago Public Schools’ double-dose algebra
• Self-concept
– Cohen’s value affirmation treatment
Impact
B or higher in 9th-grade algebra
+13% *
Passed geometry in 10th grade
+12% *
Grade 11 math scores
+.24 sd *
Graduated within 5 years
+12% *
Enrolled in any college
+11% *
Source: Cortes et al. (2011)
1.00
Mean GPA in Core Courses
Program period
0.75
0.50
0.25
0.00
-0.25
Year 1
Year 2
Cohen caveats
• Heterogeneous treatment effects: no
impacts on higher-achieving Blacks
and on Whites
• Some replication attempts show no
consistent impacts (Dee, 2014)
Pace of development in
counterfactual conditions
Null/slow => persistence?
Eventually => fadeout?
• Fractions, algebra
• Large vocabulary
• Counting
• Letters, word sounds
• Implicit theories (Dweck)
• Self-concept (Cohen)
• Academic motivation
• Implicit theories (Dweck)
• Self-concept (Cohen)
• Academic motivation
• EF working memory
• EF impulse control
• Emotional self-reg for some • Emotional self-reg for most
• Background knowledge
• Background knowledge
Distraction time by age
Time to solve conflict in ms
700
600
500
400
300
Adult
levels
200
100
0
3
4
Posner and Rothbart (2007)
5
6
7
Age
8
9
10
Distraction time by age
Time to solve conflict in ms
700
Hypothetical intervention period
600
500
400
300
Adult
levels
200
100
0
3
4
Posner and Rothbart (2007)
5
6
7
Age
8
9
10
Conditions where “eventual” development
in counterfactual may lead to fadeout
– Impulse control by age
– Vocabulary development in Head
Start
– Did the Canadian SSP accelerate
return to labor force that would
have happened anyway?
0.20
3 yr olds
0.15
4 yr olds
0.10
0.05
0.00
End of TX
One year later
Percentage Employed Full-Time
30
20
Program period
10
0
-12 -8 -4 1 5 9 13 17 21 25 29 33 37 41 45 49
-10
Months From Random Assignment
Percentage Employed Full-Time
50
40
Program period
30
20
10
Control
group
0
-12 -8 -4 1 5 9 13 17 21 25 29 33 37 41 45 49
Months From Random Assignment
Percentage Employed Full-Time
50
40
Program period
Treatment
group
30
20
10
Control
group
0
-12 -8 -4 1 5 9 13 17 21 25 29 33 37 41 45 49
Months From Random Assignment
What conditions lead to impact
persistence?
When interventions change:
• the “right kinds” of skills or capacities, or
• the “right kinds” of environments
“Right kinds” of environments:
• Malleable features of environments
that are fundamental for
promoting the “right kinds” of skills
and behaviors
• Families?
• Schools?
• Neighborhoods?
School quality can be a fundamental
environmental feature
• Winning the lottery to enter one
of NYC’s small high schools of
choice (Untermann et al., 2014)
Winning the lottery to enter a NYC Small
High School of Choice (n=14,608)
Outcomes
SSC
Control
group
Effect
Graduation
Graduated from high school 71.6
Regents diploma granted 50.2
62.2
43.5
9.4 **
6.7 **
College readiness
Passed English Regents Exam at 75+ 42.1
Passed Math Regents Exam at 75+ 25.1
35.8
24.5
6.3 **
0.5
40.7
8.4 **
Post-secondary enrollment
Enrolled in post-secondary 49.0
education
Is neighborhood quality a
fundamental environmental feature?
• Moving to Opportunity suggests
not for many outcomes in the
US
No Impacts on School Achievement
Outcome
Neighborhood Poverty Rate (~10 yr mean)
Control
mean
TOT
Impacts
40%
-18%
42
No Impacts on School Achievement
Outcome
Neighborhood Poverty Rate (~10 yr mean)
Baseline Ages 0 to 5
Reading Assessment
Math Assessment
Control
mean
TOT
Impacts
40%
-18%
ns
ns
Took SAT/ACT?
ns
Early career earnings
???
43
What else can sustain impacts?
When interventions:
• (from before) boost the “right kinds” of skills or
environments
• are supported by post-TX sustaining environments
• lead to “foot-in-the-door” access to sustaining
environments
• are intensive enough to change less-malleable skills for
children with bad counterfactual conditions
• treat enough children to generate positive peer effects
What else can sustain impacts?
When interventions:
• (from before) boost the “right kinds” of skills or
environments
• are supported by post-TX sustaining environments
• lead to foot-in-the-door access to sustaining
environments
• are sufficiently intensive to change foundational skills
for children with bad counterfactual conditions
• treat enough children to generate positive peer effects
Sustaining environments
Do higher quality K and 1st grade
classroom or home learning
environments better sustain the
impacts of the Building Blocks pre-K
math intervention?
0.70
0.60
0.50
0.40
0.30
Program
period
0.20
0.10
0.00
End of pre-K
Clements et al. 2013
End of K
Age
End of 1st
0.70
Hypothetical
sustaining
environment
0.60
0.50
0.40
0.30
Program
period
0.20
Hypothetical nonsustaining
environment
0.10
0.00
End of pre-K
End of K
Age
End of 1st
Sustaining environments?
Are BB impacts better sustained with:
• Math teaching quality in K & 1st grade
• Number of math activities K & 1st grade
• Home learning activities
• Maternal education
Jenkins, Watts, Clements et al.
(tomorrow)
0.70
0.60
0.50
0.40
0.30
Program
period
High Quality Math
Teaching
0.20
Low Quality Math Teaching
0.10
0.00
End of pre-K
Jenkins, Watts, Clements et al.
End of K
Age
End of 1st
Sustaining environments?
Are impacts better sustained with:
• “Follow through” BB training with K
and 1st grade teachers?
0.70
0.60
0.50
0.40
0.30
BB follow-through
Program
period
p =.07
0.20
0.10
0.00
End of pre-K
End of K
Age
End of 1st
Sustain environments
Building Blocks suggest that
environmental supports must be
tailored explicitly to the nature of the
prior treatment
What can sustain impacts?
When interventions:
• (from before) boost the “right kinds” of skills or
environment
• are supported by post-TX sustaining environment
• lead to foot-in-the-door access to sustaining
environments
• are sufficiently intensive to change foundational skills
for children with bad counterfactual conditions
• treat enough children to generate positive peer effects
Foot-in-the-door
Foot-in-the-door links to the emerging
literature on developmental cascades (Dodge et
al. 2008)
• Sequence of positive or negative conditions
that cumulate to good or bad outcomes
Can we avoid negative and promote positive
developmental cascades?
A cascade model of Dodge et al (2008):
Adverse
early
context
Adolescent
violence
Deviant
Peers
Early harsh
parenting
Poor school
readiness
Low parent
monitoring
Conduct
problems
School
failure
Early life
Adolescence
Foot-in-the-door examples?
• Cohen values affirmation and positive involvement in
school?
• SAT prep may affect college quality, which in turn may
have a positive impact on earnings
• Can FAFSA knowledge lead to college, which in turn
leads to a BA and later success?
• Can pro-social behavioral interventions reduce or delay
first arrests, which in turn delay later crime?
Not much evidence on longer-run outcomes
Since <100% probabilities multiply, relying on cascades seems
like a risky intervention strategy
What can sustain impacts?
When interventions:
• (from before) boost the “right kinds” of skills or
environment
• are supported by post-TX sustaining environment
• lead to foot-in-the-door access to sustaining
environments
• are intensive enough to change less-malleable skills for
children with bad counterfactual conditions
• treat enough children to generate positive peer effects
Can anything change IQ?
• What combination of intensity and
bad counterfactual conditions does it
take to sustain IQ impacts?
1.25
Abecedarian
1
Perry
0.75
Abecedarian
0.5
0.25
Perry
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
-0.25
Age
Perry vs. Abecedarian
Perry
• 1 or 2 years
• Part-day ECE + weekly
home visits
• Urban setting
• IQ ranging from 75 to 85
Abecedarian
•
•
•
•
•
5 years
Year-round full-day ECE
Cumulative curriculum
Rural setting
IQs averaging 85
• African American children
• High risk/low SES families
Infant Health and Development
Program
• Intensive center-based ECE between
ages 1 and 3, using Abecedarian
curriculum
• Low birth weight children
• One of few ECE interventions with
both low and higher-income families
IQ Impact in sd units
2.00
Low
income
1.50
1.00
Low
response
rate
0.50
High income
0.00
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18
-0.50
Duncan and Sojourner (2013)
Age
What can sustain impacts?
When interventions:
• (from before) boost the “right kinds” of skills or
environment
• are supported by post-TX sustaining environment
• lead to foot-in-the-door access to sustaining
environments
• are sufficiently intensive to change foundational skills
for children with bad counterfactual conditions
• treat enough children to generate positive peer effects
Generating peer effects
• Measles vaccinations!
• Deworming treatments in Kenya
generated large benefits for untreated
children in treatment schools (Miguel
and Kremer, 2004)
• County spending on preschool in
North Carolina?
5
More At Four Reading Scores
More At Four Math Scores
Smart Start Reading Scores
Smart Start Math Scores
4
3
2
1
Program period
Months of Learning Gained
6
0
pre-K
K
Grade 1 Grade 2 Grade 3 Grade 4 Grade 5
Outline
I. OPTIMAL CONDITIONS FOR IMPACT
PERSISTENCE
II. OTHER AVENUES FOR IMPACT
PERSISTENCE
III. HOW TO RECONCILE ECE PROGRAM
FADEOUT ON IQ WITH IMPACTS ON ADULT
OUTCOMES?
Resurrection
Death
Perry’s IQ Swan Dive and
Earnings Resurrection
1.00
.87 sd
0.75
0.50
Earnings
.44 sd
IQ
0.25
0.00
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Age
-0.25
Perry and Abecedarian both affected
adult earnings
ECE
treatment
Mediators
Adult
earnings
Do they have common (statistical) mediators?
Are they “cognitive” or “non-cognitive”?
Effect Size in sd units
1.00
0.75
IQ
0.50
0.25
0.00
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
-0.25
Age
Source: Schweinhart et al., 2005; Effect sizes >.30 are p<.05, one-tailed test
Effect Size in sd units
1.00
0.75
0.50
IQ
Language
Adult
literacy
Reading
0.25
Math
0.00
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
-0.25
Age
Source: Schweinhart et al., 2005; Effect sizes p<.05 are shaded
Index
Sample item
Impact
Academic potential
Creativity
.31ns
Academic motivation
Alert and interested; motivated;
persists
.37ns
Classroom conduct
Disobedient; impulsive; blames others
.40*
Personal behavior
Absences; swears
.36ns
Teacher dependence
Seeks constant reassurance
.03ns
Emotional state
Depressed; withdrawn
.29ns
Emotional adjustment
Trust; level of emotional adjustment
.30ns
Source: Pinto, based on Heckman et al. (2014)
1
0.75
Age 27
earnings
IQ
0.5
0.25
0
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
-0.25
Age
Solid marker denotes p<.05
Effect Size in sd units
1.00
0.75
0.50
0.25
IQ
0.00
-0.25
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
-0.50
Age
Source: Campbell et al. (2001) and Campbell et al. (2002)
Effect Size in sd units
1.00
Reading
0.75
0.50
0.25
IQ
Math
0.00
-0.25
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
-0.50
Age
Source: Campbell et al. (2001) and Campbell et al. (2002)
Effect Size in sd units
1.00
Reading
0.75
0.50
0.25
IQ
Math
Global self-worth
0.00
-0.25
-0.50
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Cognitive competence
Social competence
Age
Source: Campbell et al. (2001) and Campbell et al. (2002)
Abecedarian mediators
• Abecedarian generated a number of potential
mediational effects
– achievement and IQ
– but not a limited number of “noncognitive”
measures
Two other relevant studies
• Both the Chicago Parent-Child program
(Reynolds et al.) and kindergarten class quality
(Chetty) affected adult earnings
– CPC affected reading and math achievement more
than its “noncognitive” measures
– Chetty et al. found more impact on a noncognitive
index than on achievement
Bottom lines on impact
resurrection
An even bigger mess!
• Inconsistent evidence on what combination
of cognitive and noncognitive mediators are
at work
• “Equifinality”?
Summary
That’s it!
To help generate
discussion…
• Impact persistence requires
treating fundamental and
malleable skills that would not
develop eventually in most
counterfactual conditions
What else can sustain impacts?
When interventions:
• are supported by post-TX sustaining environments
• lead to “foot-in-the-door” access to sustaining
environments
• are sufficiently intensive to change foundational
skills for children with bad counterfactual conditions
• treat enough children to generate positive peer
effects
Greg Duncan ([email protected])
Drew Bailey ([email protected])
Winnie Yu ([email protected])

Extra slides
Bucharest Adoption Study’s impacts
on IQ
• Children in Rumanian
orphanages randomly assigned
to foster care between ages 6
and 31 months
Foster care ITT impacts on IQ
IQ impacts in sd units
0.75
0.50
p=.07
0.25
Foster care intervention period
0.00
3
4
4
5
5
6
Age
Fox et al. (2011); Nelson et al. (2007)
6
7
7
8
8
How bad are counterfactual
conditions for, say, ECE children?
% of low income 3&4 year
olds enrolled in preschool
Completed schooling of
low-income mothers
# of siblings for children in
low-income families
Then (1960s and
70s)
Now
16%
50%
8.5 yrs
11.2 yrs
3.8
1.8
=> much harder for ECE to generate large and durable
advantages now vs. decades ago
Bottom lines
• Do post-TX sustaining environment really need to be
as tailored to the treatment as in Building Blocks?
• Apart from maltreatments, have truly debilitating
environments become so rare in the US that we can’t
expect long-run miracles any more?
• For interventions involving implicit theories, selfconcept and motivation, we desperately need more
independent replications and longer-run follow-ups