Transcript Tennessee Class Size

The Effects of Charter Competition
on Academic Outcomes:
A Review of U.S. Evidence
Martin F. Lueken
Anna M. Jacob
Jennifer Ash
Prepared for the Campb ell Collaboration Colloquiu m
Copenhagen 2012
T h u r sday, M ay 3 1 2 0 1 2
INTRODUCTION
What is a charter school in the United States?
 Considered a public school
 Subject to laws that govern public schools
 More autonomous than traditional public schools
(TPS) – usually not subject to other controls (i.e.
collective bargaining agreements)
 E.g. can set own academic calendar, less restricted in hiring
decisions
INTRODUCTION
1991, Minnesota passed first charter law
in United States
Political compromise in response to push
for education vouchers
Today, 41 states with charter school laws
Charter schools serve over 1.5 million
students
BACKGROUND
Context: charter schools part of school choice
movement
 Increase school options
 Threat to traditional public schools (TPS) to lose
students, hence funding  incentive to improve
 TPS options:
 Improve teaching, how they use resources, etc. (constructive
response)
 Exert efforts to block reform, barriers to entry (non-constructive
response)
BACKGROUND
CHARTER EFFECTS
Two effects of charter schools
 Direct effect: how well do charter school students
achieve relative to TPS students?
 Indirect effect: how do other schools behave in face
of charter competition?
RESEARCH QUESTION
What is the effect of charter school
competition on student achievement in
other traditional public schools?
CHALLENGES TO SYSTEMATIC REVIEW
Analytic Challenges
Endogeneity must be addressed in charter school
studies (e.g. charter school location not random)
Outcome measures (student level vs. school level)
Variation in charter environments
 Charter laws vary significantly by state
 Some laws encourage competition, some laws impede
competition
 Funding levels, caps on # of schools or students, restriction on
locations
INCLUSION CRITERIA
How wide the net?
Definition of charter competition
 Include studies with any measure of competition
Grades
 Focus on grades K-12
Geographic level
 Include studies addressing competition up to state
level
INCLUSION CRITERIA
 Sample period: 2002 and later
 Geographic/language: United States/English
only
 Types of studies:
 only quantitative studies that attempt to account for
endogeneity problem (e.g. regressions with
instrumental variables or fixed effects)
 must include statistical control for pre-test
 Must include comparison group
 Outcomes: student scholastic achievement in
math and reading measured by standardized
exams
SEARCH STRATEGY
Phase 1: Identify Databases
Phase 2: Title Review
Phase 3: Abstract Review
Phase 4: Methods Review
Phase 5: Coding
Phase 6: Final Inclusion Decision
Phase 7: Synthesis
SEARCH STRATEGY
1.
Searched electronic databases

2.
Searched grey literature
1.
3.
NBER working papers, dissertations and theses
Hand-searched relevant journals


4.
Google Scholar, PsycINFO, ProQuest, EconLit
Journal of School Choice,
Education Next
Reviewed introduction and literature reviews of
included studies
Search results
Database
EconLit
Titles
retrieved
366
Google Scholar
788
NBER
627
ProQuest
9403
PsycINFO
730
Handsearched
74
Total
11988
Abstracts
reviewed
Methods
reviewed
Studies
coded
Studies
kept
Search results
Titles
retrieved
366
Abstracts
reviewed
88
Google Scholar
788
27
NBER
627
23
ProQuest
9403
62
PsycINFO
730
61
Handsearched
74
21
11988
282
Database
EconLit
Total
Methods
reviewed
Studies
coded
Studies
kept
Search results
Titles
retrieved
366
Abstracts
reviewed
88
Methods
reviewed
58
Google Scholar
788
27
24
NBER
627
23
6
ProQuest
9403
62
35
PsycINFO
730
61
27
Handsearched
74
21
18
11988
282
168
Database
EconLit
Total
Studies
coded
Studies
kept
Search results
Titles
retrieved
366
Abstracts
reviewed
88
Methods
reviewed
58
Google Scholar
788
27
24
NBER
627
23
6
ProQuest
9403
62
35
PsycINFO
730
61
27
Handsearched
74
21
18
11988
282
168
Database
EconLit
Total
Studies
coded
Studies
kept
22
15
Search results
Titles
retrieved
366
Abstracts
reviewed
88
Methods
reviewed
58
Google Scholar
788
27
24
NBER
627
23
6
ProQuest
9403
62
35
PsycINFO
730
61
27
Handsearched
74
21
18
11988
282
168
Database
EconLit
Total
Studies
coded
Studies
kept
22
15
LOCATIONS UNDER STUDY
Table: Locations studied in included articles
States
School Districts
Arizona (1)
Chicago
Chula Vista, CA
Florida (1)
Denver
Fresno, CA
Michigan (3)
Milwaukee
Los Angeles, CA
North Carolina (2)
New York City
Napa Valley, CA
Ohio (3)
Philadelphia
San Diego, CA
Texas (4)
San Diego
West Covina, CA
"large urban school district in SW"
MEASURES OF CHARTER COMPETITION
 Number of charter schools within a district or within
some specified distance (8)
 Enrollment shares of charter schools by district (7)
 Distance from TPS to nearest charter school (4)
 Student transfer rates from TPS to charter schools (4)
 Whether charter school is present in district (2)
CHARACTERISTICS OF 15 STUDIES
 Analytic Methods
 Fixed effects = 9
 Difference-in-differences = 3
 Instrumental variables = 3
 Level of data
 Student = 8
 School = 7
 Sources
 Peer-reviewed = 8
 Dissertations = 3
 Working papers = 2
 Reports = 2
SIMPLE VOTE COUNTING
Table: Simple vote count of studies included in systematic review
Math
Reading
Overall*
Positive
6
5
6
Mixed / no effect
5
7
7
Negative
2
1
2
*overall counts include two studies that used composite measures (positive for Holmes et al.,
2003; negative for Kamienski, 2008) -- math and reading effects could not be dissected from
these measures
ANTICIPATED CHALLENGES
Shocks to life-as-usual
ANTICIPATED CHALLENGES
Challenges in gathering data from studies
 Which estimates to include?
 Numerous models and robustness checks run
 Some studies (i.e. Zimmer & Buddin, 2009) estimate effects
separately for elementary, middle, and high schools; others
(i.e. Sass, 2006) produce an aggregate estimate for all
grades
 Outcome measures?
 Most studies use individual student test scores
 Some studies (school-level data) use schools’ proficiency
rates as outcomes
 How to compute effect size? Two separate ones?
CONCLUSIONS
Currently planning how to best meta-analyze
data
Potential moderator analyses
 Effect sizes by states
 Effect sizes by district level
 Effect sizes by racial background
CONTACT
Martin F. Lueken
University of Arkansas
[email protected]
Anna M. Jacob
University of Arkansas
[email protected]
Jennifer Ash
University of Arkansas
[email protected]