Transcript Education Data - University of Utah

Statistical Modeling
for Education Planning
[email protected]
www.schools.utah.gov/finance
URBPL 5/6020 / April 19, 2007
Who We Are

Utah State Office of Education
–


Staff to the State Board of Education
Financial and Business Services Division
Finance and Statistics Section
What We Deal With

Populations
–
–
–

Finance
–
–
–

Students
Staff
Schools
Minimum School Program (MSP) Budget & NCLB Allocations
Financial Reporting & Auditing
Property Tax
Operations
–
–
–
School Facilities
Student Transportation
Safety
How We Do It

Acquire
–

Allocate
–

Data
Money to local education agencies according to
data
Audit
–
For accuracy of data and appropriateness of
expenditures
Analytics Cycle

Population (Enrollment) Projections
–

Fiscal Impact Analysis
–

How do we get the right amount to the right place?
Compliance Audit
–

How much will service options cost?
Formula Allocation
–

How many people do we need to serve?
How well did service providers follow the rules?
Program Evaluation
–
How well did we serve the population?
Enrollment Projections:
Institutional Context

Common Data Committee
–
–
–

Current Work
–
–
–
–

Legislative Fiscal Analyst
Governor’s Office of Planning and Budget
Utah State Office of Education
By county (then allocate to districts and adjust for charter schools)
In October
Single year (to next October)
Agreed upon figures for legislative session
Future Plans
–
Multiyear project with GOPB using REMI for Baseline 2008
Enrollment Projections:
Model
Cohort Progression
 Participation Ratio

–
Kindergarten subset
Enrollment Projections:
Data

Historical Variables
–
–
–
–

Total Enrollment (Current and Prior Years)
Grade 12 Enrollment (Current and Prior Years)
Kindergarten Enrollment (Current Year)
Births (by Month, 4- and 5-Years Prior)
Intermediate Variable
–
Projected Kindergarten Enrollment
Enrollment Projections:
Formula
Formula
EY+1
= EY
+ (BY-4 * (KY / BY-5 )) - GY)
+ (EY - EY-1) - (KY - GY-1)
Element
Base Population
Cohort Progress.
Implied Migration
Fiscal Impact Analysis:
Example

HB 222 (2002)
–


“Make recommendation on … the ideal size of
schools districts in this state …”
Optimization Problem
Cost Function
–
Relates expenditures per student to enrollment
(main cost driver) controlling for academic
achievement (output measured in quality)
Fiscal Impact Analysis:
Design
Sample:
Data:
Model:
Procedure:
Fit (adj R2):
Predictors:
Coeff (b):
Sig (p):
Cross section of 40 Utah school districts
Superintendent’s Annual Report, 2000-01
Y = m + (b1X + b2X2) + b3Z + e
OLS regression
.29
Enroll
Enroll2
Lexile
-.138
.0000016
-665
.01
.05
.03
Fiscal Impact Analysis:
Results

Empirical Cost Function
–

Differentiated and Set Equal to Zero
–

exp = .0000016enr2 - .138enr – 665lex + 6,468
0 = .0000032enr - .138
Solution is Optimal Size
–
enr = .138/.0000032 = 43,125 students
Fiscal Impact Analysis:
Politics (TTC, SL Tribune 2/23/04)

Columbia University professor’s critique:
–

Fiscal analyst’s defense:
–

“Anybody’s guess is as good as the next person’s”
Opponents’ critique:
–

“I’d be happy to go with the [USOE] analysis rather than the
fiscal analyst’s, which is opaque to the point of
incomprehensibility”
“Foes have long accused the fiscal analyst’s office of
working the numbers to achieve a favorable outcome”
Fiscal analyst’s concession:
–
“At the outset, the intention is to have it come out in a
positive way so there’s not a cost”
Fiscal Impact Analysis:
Ethics


Substantive claims must be warranted by
evidence
Production of evidence must be based on
transparent procedures
Allocation Formulas:
Minimum School Program (1)



“The purpose of this chapter [Utah Code 53A-17a] is
to provide a minimum school program (MSP) for the
state in accordance with the constitutional mandate.”
“It recognizes that all children of the state are entitled
to reasonably equal educational opportunities
regardless of their place of residence in the state
and of the economic situation of their respective
school districts or other agencies.”
NOTE: Overriding concern with equity; adequacy is
another issue of growing legal importance, but its
operationalization is very unclear.
Allocation Formulas:
Minimum School Program (2)


“It further recognizes that although the establishment
of an educational system is primarily a state function,
school districts should be required to participate on a
partnership basis in the payment of a reasonable
portion of the cost of a minimum program.”
NOTE: Utah sources of revenue (FY 2006):
–
–
–
State 55% from income tax
Local 36% from property tax
Federal 9% from who knows where
Allocation Formulas:
Minimum School Program (3)


“Each locality should be empowered to
provide educational facilities and
opportunities beyond the minimum program
and accordingly provide a method whereby
that latitude of action is permitted and
encouraged.”
NOTE: Local school boards can impose
several additional property taxes for specified
educational purposes.
Allocation Formulas:
Budgeting for Basic Program (1)




Majority of funding is based on “Prior Year +
Growth” formula
Prior year is Average Daily Membership
(ADM)
Growth is percentage difference between
projected Fall Enrollment and current year
Fall Enrollment
Hold harmless in case of negative growth
Allocation Formulas:
Budgeting for Basic Program (2)



Result is number of Weighted Pupil Units
(WPUs), a quantification of the basic service
which each local education agency is
obligated to provide
Legislature sets monetary value of WPU
every year
Total WPUs times WPU $ value determines
basic appropriation
Allocation Formulas:
Budgeting for Basic Program (3)



If LEA property tax revenue cover its
obligation, then buck stops there; otherwise,
state pays balance from income tax revenue
In practice, all LEAs need some state
assistance and appropriations often fall
short, so funds are prorated according to
WPUs
Since FY 2001, K-12 funding has
approximately kept pace with inflation
Allocation Formulas:
Categorical Programs


In addition to the basic program, the
Legislature has established dozens of
categorical programs to address particular
concerns
Cost drivers of categorical programs can be
quite complex — “Special Education Add On”
is an especially striking example of what can
happen when trying to reconcile competing
interests through a funding program
Allocation Formulas:
Categorical Program Example (1)

Per WPU, which is the greater of the
average of Special Education (Self
Contained and Resource) ADM over the
previous 5 years (which establishes the
foundation [hold harmless] below which the
current year WPU can never fall) or prior
year Special Education ADM plus weighted
growth in Special Education ADM.
Allocation Formulas:
Categorical Program Example (2)

Weighted growth is determined by multiplying
Special Education ADM from two years prior
by the percentage difference between
Special Education ADM two years prior and
Special Education ADM for the year prior to
that, subject to two constraints:
Allocation Formulas:
Categorical Program Example (3)


Special Education ADM values used in
calculating the difference cannot exceed the
prevalence limit of 12.18% of total district
ADM for their respective years.
If this measure of growth in Special
Education exceeds current year growth in
Fall Enrollment, growth in Special Education
is set equal to growth in Fall Enrollment
(incidence limit).
Allocation Formulas:
Categorical Program Example (4)



Finally, growth is multiplied by a factor of
1.53.
This weight is intended to account for the
additional cost of educating a special
education student.
However, the weight is not based on an
empirical analysis of the cost of special
education relative to "regular" education.
An Australian Approach:
Victoria’s Principles (1)

Preeminence of Educational Considerations
–

Elimination of disparities reflecting historical and
political decisions for which there is no current or
future educational rationale
Cost Effectiveness
–
Relativities among allocations should reflect
knowledge of efficient ways of achieving school
and classroom effectiveness
An Australian Approach:
Victoria’s Principles (2)

Fairness
–

Schools with the same mix of learning needs
should receive the same total resources; this
requires accurate and comprehensive information
on those student characteristics which best
predict academic achievement
Transparency
–
Basis for allocations should be made public and
readily understandable by all with an interest
An Australian Approach:
Victoria’s Principles (3)

Subsidiarity
–

Decisions on resource allocation should be made
centrally only if they cannot be made locally
Accountability
–
A school that has authority to make decisions on
how resources will be allocated should be
accountable for the use of the resources,
including educational outcomes in relation to
learning needs
An Australian Approach:
Simple Budget Structure (1)

Core Funding
–
–

Grade Level
School Size
Student Disadvantage
–
–
–
–
Disabilities
Special Learning Needs
English as Second Language
Rurality and Isolation
An Australian Approach:
Simple Budget Structure (2)



Facilities (operation & maintenance)
Administration
Costs outside of control of schools
–

e.g., Transportation to and from school
“Priority” Programs
–
Money for politicians of the day to play with
An Australian Approach:
Special Learning Needs



Sample (83 schools; 7,233 students)
Hierarchical linear & Structural equation modeling
Demographic index to predict achievement:
–
–
–
–
–
–
Poverty (qualified for education welfare payment)
Parental occupation (skill level)
Language spoken at home (other than English)
Family composition (two parent, one parent, none)
Aboriginality (= Alaska Native or American Indian)
Transience (recently changed schools, = Mobility)
An Australian Approach:
Reference


Hill, Peter W. (1996). Building equity and
effectiveness into school based funding
models: An Australian case study. 18p.
http://nces.ed.gov/pubs97/97535i.pdf
Compliance Audit:
Purpose



Provide reasonable assurance that local
education agencies are correctly applying
State Board of Education rules in accounting
for their students
Statistical summaries from individual data
files serve as written management assertions
Auditors follow agreed upon procedures
Compliance Audit:
Sampling



Efficient auditing depends on selection of
sample appropriate to purpose
For example, if you want to adjust statistics
based on audit, you need a probability
sample
The right sample size then depends on:
–
–
Variation in the population
Risk you are willing to take of being wrong
Sample Size: The Price of Precision
pop = 80,000; mean = 154; sd = 25
90%
95%
99%
1%
703
1001
1718
5%
29
41
71
10%
8
11
18
Program Evaluation:
Regression with Treatment as Dummy


“In the actual practice of applied social
science, the most common mode of causal
inference, the most common quasiexperimental design …” (Cook & Campbell)
Crucial to valid interpretation:
–
–
Specification of correct theory (of nonrandom
selection process) as represented by equation
(Near) perfect measurement of variables
Program Evaluation:
Recommendation

Consider path analysis as extension of
regression:
–

Consider multiple indicators of each variable:
–

Explicit theory of how program works as a causal
mechanism in form of path diagram
Use factor analysis to obtain composite measure
representing only common variance
In short, poor man’s structural equation
modeling
Program Evaluation:
Path Diagram Example
Program Evaluation:
Bibliography

Fitzpatrick, Sanders & Worthen (2004) Program Evaluation:
Alternative Approaches and Practical Guidelines
–

Patton (1997) Utilization Focused Evaluation
–

H97 .M64
Cook & Campbell (1979) Quasi Experimentation: Design and
Analysis for Field Settings
–

H62.5.U5 P37
Mohr (1995) Impact Analysis for Program Evaluation
–

LB2822.75 .W67
H62 .C5857
Scriven (1991) Evaluation Thesaurus
–
AZ191 .S37
Some Education Data Issues





Is a Navajo living in a hogan homeless?
Kanab is on the urban fringe of which city?
Who decides the racial identity of a student?
When is a person who leaves school without
graduating not a dropout?
Is being in a single parent family a reliable
indicator of being at risk of low academic
performance?
Highly Impacted Schools Criteria:
Factor Analysis
% of Enrollment Median Loading Median Loading
Ethnic Minority
.93
.95
Limited English
.91
.93
Free Lunch
.80
.86
Single Parent
-.49
-
Mobile
-.60
-
R2
55%
82%
Data Sources




Digest of Education Statistics
http://nces.ed.gov/programs/digest
NCES Tables & Figures
http://nces.ed.gov/quicktables
USOE Assessement, Accountability & Division
http://www.schools.utah.gov/eval
Utah State Superintendent’s Report
http://www.schools.utah.gov/finance/other/AnnualRe
port/ar.htm