Transcript Melbourne Institute Powerpoint Template

Early Bird Catches the Worm: The Causal Impact of Pre-school
Participation and Teacher Qualifications on Year 3 NAPLAN
Cognitive Tests
Diana Warren and John Haisken-DeNew
ACER Research Conference
4 August 2013
Funding for this research is gratefully acknowledged by the research partnership between the Melbourne Institute of Applied Economic and Social Research and the
Victorian Department of Education and Early Childhood Development (DEECD). This paper uses unit record data from the Longitudinal Study of Australian Children
(LSAC) Survey. The LSAC project was initiated and is funded by the Australian Government Department of FaHCSIA and is managed by the Australian Institute of Family
Studies (AIFS). The findings and views reported in this paper, however, are those of the authors and should not be attributed to either FaHCSIA or AIFS.
Key Research Questions
 How does attendance
at pre-school or
kindergarten in the year
prior to formal schooling
affect NAPLAN Scores
in Year 3?
 Do the benefits from
pre-school differ
according to the
qualification of the
pre-school teacher?
www.melbourneinstitute.com
Method and Key Findings
 Data: The Longitudinal Study of Australian Children (LSAC).
 Identification Strategy: Kernel Based Propensity Score Matching.
 Key Findings:
 Significant positive association between pre-school or kindergarten
attendance and Year 3 NAPLAN Scores.
 The impact of pre-school is substantial: Around 20 NAPLAN points,
or 30% to 40% of one year of schooling.
 Teacher qualifications are important: Children whose pre-school
teacher had a degree or diploma specialising in early childhood
education gained the most from pre-school.
www.melbourneinstitute.com
The Importance of Early Childhood Education
 The years from birth to age 5 have been identified as the most important
developmental period during childhood.
 Critical periods in early childhood during which particular skills and
abilities are more readily acquired (Knudsen et. al., 2006).
 Brain development in the first years of life lays the foundation for:
 Language development and literacy acquisition
 Cognitive processes
 Emotional development
 Self-regulation and problem-solving skills
and has a lasting impact on health, future learning and life success
(McCain and Mustard, 1999; Shonkoff and Phillips, 2000).
www.melbourneinstitute.com
Substantial Returns from Investment in Pre-school
 As capacity for change in human skill development is highest early in life
and decreases over time, the returns on public investment in high quality
early childhood education are substantial (Heckman et. al 2006).
 Returns to the individual in terms of increased earnings, higher
education, improved physical and mental well-being.
 Positive returns to society in terms of reduced crime and delinquency,
public expenditure savings and increased tax revenues.
 As learning is a cumulative process in which early skills facilitate further
skill acquisition, the benefits of early interventions are larger and are
enjoyed for longer (Heckman, 2006).
 Early intervention programs are often more cost effective than later
remediation (Carniero and Heckman, 2003).
www.melbourneinstitute.com
Rate of Return Declines as Children Get Older
Source: Heckman (2006)
 The rate of return from investment in human capital declines as children
get older. Remedial programs in the adolescent and young adult years
are more costly in producing the same level of skill attainment in adulthood
(Heckman, 2006).
www.melbourneinstitute.com
Benefits of Targeted Pre-school Programs
 Significant benefits for children who attend high quality pre-school
programs:
 Better intellectual development
 Higher levels of concentration, sociability and independence
 Some of the best known evidence of the benefits of high quality early
education experiences on later development comes from targeted early
intervention programs undertaken in the United States.
 These programs are intended for disadvantaged children, particularly
those from single-parent or low-income families.
www.melbourneinstitute.com
The HighScope Perry Pre-school Program
 Targeted towards children from low-income households who were
assessed to be at high risk of school failure.
 A 2-year program of weekday morning pre-school routine combined
with weekly home visits by program staff.
 Teacher-child ratio of one to five, with teachers qualified in early
childhood education.
 Classrooms arranged to support children’s self-initiated learning
activities as well as small-group and large-group activities.
 Children who participated in the program were better prepared for
school and significantly outperformed the no-program group on
various intellectual and language tests from their pre-school years up
to age 14, and also on literacy tests at ages 19 and 27.
www.melbourneinstitute.com
The Abecedarian Project
 An early intervention program administered to low-income children in the
United States, with long-lasting benefits.
 Infants were randomly assigned to either a treatment or control group.
 All of the children received child care and health services at a centre with
enriched resources.
 The treatment group received intense intervention, consisting of a
pre-school and school-age education program.
 Statistically significant difference in the IQ of children in the treatment
group at the end of pre-school and also at the ages of 8 and 12.
 The children who participated in the intervention program completed
more years of education, and were more likely to attend a four-year
college.
www.melbourneinstitute.com
Head Start
 Targeted to local community needs and cater for several hundred
thousand children across the United States.
 Half-day, centre-based programcare for children from disadvantaged
families in the two years before starting school.
 Program participants may also receive related health, parenting and
other child and family support services.
 Significant short-term increases in test scores, as well as reductions in
grade repetition and learning disability diagnosis, particularly for the most
disadvantaged male children in the sample.
 In the long-term,
 Male Head Start participants were less likely to be out of the labour
force
 Female Head Start participants were more likely to attend and
complete one year of college.
www.melbourneinstitute.com
The Chicago Child Parent Centres (CPC)
 A part-day public kindergarten program for children aged 3 and 4 who
are at risk of academic underachievement.
 Follow on program providing reduced class sizes and encouraging
parental involvement during the first three years of school.
 Run by teachers with college degrees and early childhood certification,
emphasising basic skills in language and mathematics.
 At age 28, CPC participants had:
 Higher levels of educational attainment
 Higher income levels
 Significantly lower rates of drug and alcohol abuse
 Lower rates of crime
www.melbourneinstitute.com
Large Scale Public Pre-school Programs
 Larger scale public programs often have weaker effects than targeted
programs (Barnett, 1998; Dumas and Lefranc, 2010).
 Mixed evidence about the long-term effects of typical pre-school
programs:
 Long-term academic and social benefits for all children (Berlinski,
Galiani and Manacorda, 2008).
 Substantial gains for disadvantaged children (Fitzpatrick, 2008).
 Academic benefits of pre-school attendance tend to fade over time
(Goodman and Sianesi, 2005).
 Pre-school attendance may be associated with poorer behavioural
outcomes (Magnuson, Ruhm and Waldfogel, 2007).
 No conclusive evidence that a pre-school teacher with a Bachelor degree
will ensure better cognitive outcomes (Early et. al, 2007).
www.melbourneinstitute.com
Early Childhood Education in Australia
 Australian families are offered a diverse range of options.
 Each state and territory offers non-compulsory pre-school education to
children in the year prior to their first formal year of schooling.
 Programs are provided in a mix of contexts, including stand-alone preschools, kindergartens, Long Day Care, and early learning centres.
 Until recently, there were no nationally agreed or consistent standards for
staffing across the child-care and pre-school sector.
 As part of the 2008 COAG agreement, all children in the year before
starting formal schooling now have access to an early childhood education
program
 Delivered by a degree qualified early childhood teacher
 15 hours per week, 40 weeks per year
www.melbourneinstitute.com
Costs and Incentives
 Incentives provided by State and Federal governments:
 Additional university places for early childhood education degrees
 Removal of TAFE fees for diploma-level qualifications
 Fee remissions for early childhood teachers willing to work in regional
areas
 Of the $1.4 billion expenditure by state and territory governments on
ECEC in 2012-13, 80% ($1.1 billion) comprised expenditure on pre-school
services (Productivity Commission, 2014).
 Given the substantial cost of public pre-school programs, it is of particular
policy interest to examine the longer-run impacts of these programs.
 There are very few studies examining the impact of early childhood
education and care in Australia.
www.melbourneinstitute.com
Pre-school Literacy Project (PLP)
 40 pre-schools across Victoria between 1996 and 1999.
 Pre-school teachers encouraged to introduce literacy into their programs:
 Placing a writing table in the room along with appropriate resources
 Introducing literacy materials into the “home corner”
 Introducing a post box for letter exchanges
 Bringing the print around their room down to the children’s eye level
 After one year of primary school, PLP students had:
 Significantly higher scores on reading and writing tests
 Higher level oral language skills
 More sophisticated phonological awareness
 During the second year of school, the PLP students maintained their
advantage in reading and oral language proficiency.
www.melbourneinstitute.com
Child Care Choices Longitudinal Extension Study
 A study of the child care and early school experiences of children in urban
and rural New South Wales from 2002 to 2008 (Bowes et. al., 2009).
 Hours of care, multiple and changeable care arrangements and the quality
of the carer-child relationship were important predictors of children’s
achievement.
 Longer hours in early formal child care were found to be associated with
poorer academic achievement.
 Longer hours of early informal care had a positive effect on social
behaviour.
www.melbourneinstitute.com
Performance Indicators in Primary Schools (PIPS)
 The Performance Indicators in Primary Schools (PIPS) program assesses
the early literacy and numeracy skills of students.
 PIPS is used to assess each student’s early literacy and numeracy skills
and identify students who will benefit from intervention or enrichment
programs.
 Boardman (2005) compared the PIPS scores of 884 students across 38
schools in Tasmania, who began their preparatory year of school in
January 2004.
 Reading, Numeracy and overall test scores were significantly higher
among children who had attended full-day sessions of kindergarten the
previous year.
www.melbourneinstitute.com
Australian Studies using the LSAC Data
 Harrison (2009):
 Children who were attending an early childhood or pre-school
program at the age of 4 or 5 were more competent in language
ability.
 Vocabulary scores were negatively associated with longer weekly
hours at child care or pre-school, particularly among children who
were in care for more than 30 hours per week.
 Claessens (2009):
 Examined the association between general cognitive ability and
socio-emotional skills at the age of four or five and academic
achievement four years later.
 Cognitive ability at the age of 4 or 5 is an important predictor of
achievement in middle childhood.
www.melbourneinstitute.com
The Longitudinal Study of Australian Children (LSAC)
 Waves 1 and 3 of the Longitudinal Study of Australian Children (LSAC).
 The first wave of interviews began in March 2004 and families are
subsequently interviewed every two years.
 Cognitive outcomes are measured using NAPLAN Scores for Numeracy,
Reading, Spelling, Writing and Grammar.
 Sample: 2229 children who had not yet begun formal schooling in 2004,
with matched NAPLAN information for Year 3 in 2008.
 Over 90% of children attended some type of pre-school or kindergarten
program in the year before starting school.
www.melbourneinstitute.com
NAPLAN Bands
www.melbourneinstitute.com
Average NAPLAN Scores, by Pre-school Attendance
440
420
423
434
430
428
422
408
400
396
398
398
Reading
Spelling
405
380
360
340
320
300
Numeracy
Preschool
Writing
Grammar and
Punctuation
No Preschool
www.melbourneinstitute.com
NAPLAN Score Distributions, Year 3 Numeracy
.006
Year 3 Numeracy
.003
0
.001
.002
Density
.004
.005
preschool
no preschool
0
100
200
300
400
500
NAPLAN Band (%)
3
4
1
2
Pre-school
1.6
5.5
18.7
No Pre-school
*4.3
*7.5
29.0
600
700
800
5
6
Total
26.9
25.5
21.0
100.0
23.2
25.7
10.4
100.0
www.melbourneinstitute.com
NAPLAN Score Distributions, Year 3 Reading
.006
Year 3 Reading
.003
0
.001
.002
Density
.004
.005
preschool
no preschool
0
100
200
300
400
500
NAPLAN Band (%)
3
4
1
2
Pre-school
3.3
9.0
13.6
No Pre-school
*5.2
13.1
22.4
600
700
800
5
6
Total
19.8
28.6
25.8
100.0
20.8
21.8
16.6
100.0
www.melbourneinstitute.com
NAPLAN Score Distributions, Year 3 Spelling
.006
Year 3 Spelling
.003
0
.001
.002
Density
.004
.005
preschool
no preschool
0
100
200
300
400
500
600
700
800
NAPLAN Band (%)
1
2
3
4
5
6
Total
Pre-school
2.7
7.9
13.6
26.9
27.0
21.9
100.0
No Pre-school
*4.2
13.6
23.9
26.6
15.7
16.0
100.0
www.melbourneinstitute.com
NAPLAN Score Distributions, Year 3 Writing
Year 3 Writing
.004
.003
0
.001
.002
Density
.005
.006
preschool
no preschool
0
100
200
300
400
500
600
700
800
NAPLAN Band (%)
1
2
3
4
5
6
Total
Pre-school
2.2
4.1
13.4
26.2
27.7
26.4
100.0
No Pre-school
3.2
6.8
19.0
30.6
21.7
18.7
100.0
www.melbourneinstitute.com
NAPLAN Score Distributions, Year 3 Grammar
.006
Year 3 Grammar and Punctuation
.003
0
.001
.002
Density
.004
.005
preschool
no preschool
0
100
200
300
400
500
600
700
800
NAPLAN Band (%)
1
2
3
4
5
6
Total
Pre-school
2.5
6.1
14.4
25.7
19.2
32.1
100.0
No Pre-school
6.1
7.4
22.2
25.5
17.8
20.9
100.0
www.melbourneinstitute.com
Characteristics of Children who Attended Pre-school
 Simple differences in test scores should not be regarded as causal
effects of pre-school attendance.
 They may reflect other characteristics that are correlated with both preschool attendance and NAPLAN outcomes.
 Children who did not attend pre-school were less advantaged than those
who did attend.
 More likely to be in low-income and lone parent households; children
whose parents did not complete high school were less likely to attend
pre-school.
www.melbourneinstitute.com
Estimating the Effects of Pre-school Attendance
 OLS estimates: Average effect of pre-school participation on NAPLAN
scores.
 Controlling for characteristics of :
– the child (gender, age, ATSI status, birth weight, health)
– the household (household income, household size, lone parent
household, older and younger siblings, LOTE)
– the mother (education, age, employment status); and
– state and region of residence
 To control for the innate ability of the child, models are re-estimated with
the child’s score on the “Who Am I?” test, taken at the time of their 2004
interview.
www.melbourneinstitute.com
The “Who Am I?” Test
 A measure of cognitive development created by the Australian Council
for Educational Research (ACER) in 1997.
 Developed based on previous research about the use of copying and
writing tasks for the assessment of children’s developmental level and
school readiness.
 Children are asked to write their name, copy and write letters, numbers,
words and sentences with simple instructions and encouragement from
their interviewer.
 These abilities have been shown to be strongly associated with cognitive
development and subsequent school achievement.
 Provides a reliable measure of development which is valid across cultural
groups and among children whose knowledge of English is limited.
www.melbourneinstitute.com
OLS Estimates, Effect of Pre-school on NAPLAN Scores
Without Control
With Control
for Ability
for Ability
Numeracy
13.6**
9.9**
Reading
18.7***
14.7**
Spelling
17.3**
13.7***
Writing
10.9**
7.8
Grammar
14.4**
9.9
Note: ***, ** and * represent statistical significance at the 1%, 5% and 10% levels respectively.
www.melbourneinstitute.com
OLS Estimates: Characteristics of the Child
 Boys score higher than girls in Numeracy.
 Girls score higher than boys in Writing, Spelling, and Grammar.
 Children of Aboriginal and Torres Strait Islander background have
substantially lower scores for Spelling, Grammar and Numeracy.
 Children who speak a second language have significantly higher
Spelling scores.
 Children who weighed less than 2.5kg at birth scored lower in Reading,
Grammar and Numeracy.
 Average test scores increase by 2 to 3 points for each additional month
of a child’s age.
www.melbourneinstitute.com
OLS Estimates: Household Characteristics
 Children who had an older resident sibling had lower average test
scores in Reading, Spelling and Grammar.
 Children who had younger resident siblings had significantly higher
scores for Grammar.
 Across all five domains, there was a small but significant effect of
household income, with average test scores increasing by
approximately 1 point with every $100 of weekly household income.
www.melbourneinstitute.com
OLS Estimates: Characteristics of the Mother
 Mother’s education level is a very important factor.
 Compared to children whose mother had not completed high school,
average test scores of children whose mother had a degree-level
qualification were significantly higher:
 Numeracy: 33 points
 Reading:
41 points
 Spelling:
22 points
 Writing:
30 points
 Grammar: 36 points
 This may be due to innate ability, a better home learning environment or
parents placing more value on education.
www.melbourneinstitute.com
Propensity Score Matching
 Children are divided into two groups:
 The treatment group (who attended pre-school)
 The control group (who did not attend pre-school)
 Each child in the treatment group is matched as closely as possible with
a child in the control group based on their observable characteristics.
 Taking the difference in the mean NAPLAN scores between the set of
students who attended pre-school and the observationally equivalent
matched set of students who did not attend pre-school.
 Propensity score matching can provide causal estimates, assuming that
both the treatment assignment and the outcome of interest do not depend
on unobservable characteristics.
www.melbourneinstitute.com
Average Treatment Effects
 Average effect of Treatment on Treated (ATT):
 The benefit from pre-school attendance for those who attended.
 Average effect of Treatment on Untreated (ATU):
 How much higher the NAPLAN scores of children who did not go
to pre-school might have been, if they had attended.
 Treatment effects are estimated with and without the inclusion of the
“Who Am I?” Score; and including variables measuring the home
learning environment when the child was 4 years old:
 How often the child was read to each week
 How many children’s books in the child’s home
 At home and out of home activities (e.g. music, sports)
www.melbourneinstitute.com
Causal Estimates of Pre-school Participation
Without control
for ability
ATT
ATU
With control
for ability
ATT
With controls for
ability and home
learning
environment
ATU
ATT
ATU
Numeracy
18.2***
19.3***
15.2***
16.6***
12.4*
13.1***
Reading
20.9***
24.9***
18.0***
21.1***
16.1*
18.1***
Spelling
20.2***
21.6***
17.5***
17.6***
15.9**
17.0***
Writing
13.1**
15.8***
10.1*
12.0***
7.7
10.5*
Grammar
12.1
20.2***
9.6
15.5***
6.4
12.2**
Note: ***, ** and * represent statistical significance at the 1%, 5% and 10% levels respectively.
www.melbourneinstitute.com
The Role of Pre-school Teacher Qualifications
 In 2004, most children attended some type of pre-school or kindergarten
program, but it is likely that there were substantial differences in the
quality of pre-school programs.
 No nationally agreed or consistent standards for staffing across the child
care and pre-school sector.
 Models are estimated to compare the effects of specific pre-school
teacher qualifications:
Qualification of pre-school teacher
%
Degree in Early Childhood Education or Child Care
47.5
Other Teaching Degree
9.8
Diploma in Early Childhood Education or Child Care
24.9
Certificate in Early Childhood Education or Child Care
10.2
Other (No relevant Child Care or Teaching Qualification)
7.5
www.melbourneinstitute.com
Causal Estimates: Average Treatment Effect on Treated
Numeracy
Without Control for Ability
Reading
Spelling
Writing
Grammar
Early Childhood Education Degree
15.2**
19.4***
17.5**
8.2
6.5
Other Teaching Degree
18.4*
22.6*
19.2
19.5*
15.5
Diploma
21.2***
23.6***
20.1**
9.7
15.0
Certificate
12.6
17.4
18.1
7.81
2.4
No relevant qualification
14.6
14.9
16.9*
10.2
12.3
Writing
Grammar
Numeracy
With Control for Ability
Reading
Spelling
Early Childhood Education Degree
12.5*
17.1**
15.0**
6.3
4.5
Other Teaching Degree
15.5
19.3
16.3
16.6
11.9
Diploma
16.5**
18.6**
15.4*
6.4
10.6
Certificate
12.4
17.3
18.7
8.5
1.0
No relevant qualification
14.2
13.9
17.5
8.8
10.8
Note: ***,
**
and * represent statistical significance at the 1%, 5% and 10% levels respectively.
www.melbourneinstitute.com
Conclusions
 These are the first results for Australia that show a significant impact of preschool attendance on later NAPLAN outcomes.
 Pre-school is Important:
 For Reading, Spelling and Numeracy, causal ATT effects of 17 to 20
points.
 After controlling for ability, estimated effects are reduced by 2 to 4
points.
 These causal estimates are substantial, with pre-school amounting to
30-40% of the learning impact of one year of schooling, 3 years later.
 Pre-school Teacher Qualifications and Specialisation are Important:
 Children whose pre-school teacher had a diploma or degree in Early
Childhood Education or Child Care gained the most.
 These results contrast Early et. al (2007) who find no association
between teacher qualification and outcomes in the pre-school year.
www.melbourneinstitute.com
Policy Implications
 The long-run causal impacts of pre-school attendance confirm the value
and importance of high quality pre-school programs for later cognitive
outcomes.
 For maximal program impact, pre-school teachers should have at least a
diploma level qualification.
 The COAG agreement ensuring all children have access to a pre-school
program delivered by a degree qualified early childhood teacher is likely to
have substantial long-term benefits.
 The Productivity Commission Draft Report into Early Childhood Education
and Care (2014) recommends that:
“The Australian Government should continue to provide per child
payments to the states and territories for universal access to a preschool program of 15 hours per week for 40 weeks per year.”
www.melbourneinstitute.com
Thank You
The complete paper is available at:
http://www.melbourneinstitute.com/downloads/working_paper_series/wp2013n34.pdf
www.melbourneinstitute.com