Transcript Slide 1

Data Reporting and Evaluation: Back
to the Basics
Liann Seiter and Dory Seidel, NDTAC
Agenda
• Collection of Data
– Federal collection requirements
– State collection processes
– Data quality
– Resources available for collection
• Reporting of Data
– Federal data collection systems
– Federal reporting timeline
– Resources available for reporting
• Evaluation
– Federal use of data
– State use of data
• Beyond the Basics
ND Coordinator responsibilities
related to data collection and
reporting are highlighted in text boxes
2
Collection of Data
Federal collection requirements
State collection processes
Data quality
Resources available for collection
3
Collection: Statutory Requirements
Each State agency and local educational agency shall—
(1) submit evaluation results to the State educational agency and the Secretary; and
(2) use the results of evaluations under this section to plan and improve subsequent
programs for participating children and youth.
State and local agencies receiving Title I, Part D, funds must evaluate their programs’
impact on the ability of students:
(1) to maintain and improve educational achievement;
(2) to accrue school credits that meet State requirements for grade promotion and
secondary school graduation;
(3) to make the transition to a regular program or other education program operated
by a local educational agency;
(4) to complete secondary school (or secondary school equivalency requirements)
and obtain employment after leaving the correctional facility or institution for
neglected or delinquent children and youth; and
(5) as appropriate, to participate in postsecondary education and job training
programs.
Source: Title I, Part D, Statute, Subpart 3
Become familiar with
statutory requirements
4
Collection: Two Federal Data Collections
Major Federal data collections for Title I, Part D:
• Annual Child Count
– Used by U.S. Department of Education (ED) to
determine Title I, Part A, and Title I, Part D, funding
allocations
• Consolidated State Performance Report (CSPR)
– Provides demographic and outcome data for
programs receiving Title I, Part D, funds
Understand the types of data
requested by ED
5
Collection: Federal Collection Requirements
All Title I, Part D-funded programs must collect
data to report to ED.
Identify the programs
receiving funds
Program categories:
• At-Risk Programs—Subpart 2 only
• Other Programs
• Neglected Programs
• Juvenile Detention
• Juvenile Corrections
• Adult Corrections—Subpart 1 only
Resources: CSPR Forms, CSPR Guide
6
Collection: Federal Collection Requirements
Each State agency and local agency program
submits the following data to ED:
• Student and facility counts
• Student demographics
Be familiar with the
CSPR measures
• Transition services
• Academic and vocational outcomes
• Academic performance in reading and
mathematics
7
Collection: State Collection Processes
• State and Federal reporting
requirements may differ.
• Each State has its own collection
tool and processes (i.e., paper and
pencil, online tool, electronic files).
• Data contacts at State, subgrantee,
and program levels are responsible
for certain aspects of data collection
and reporting.
Know State
reporting
requirements
Develop systems to
assist and improve
the data collection
process
Build relationships
with the data
contacts in the State
8
Collection: Data Quality Review
• Data need to be trustworthy—high-quality
data are accurate, consistent, unbiased,
understandable, transparent.
• Data quality is best achieved when timely
reviews are done at the program level.
Assess the quality of
the data received
Provide training to
improve data quality
• Educating subgrantees/programs on data
collection can improve data quality.
9
Collection: Data Quality
ED
SEA
State
Agency
S1
A
LEA 1
S2
B
1A
LEA 2
S2
1B
2A
10
Collection Resource: CSPR Forms
http://www2.ed.gov/admins/lead/account/consolidated/index.html
11
Collection Resource: CSPR Guide
“Instructional Guide to Reporting Title I,
Part D, Data in the CSPR”
•
•
•
•
Overview of data submission
Table-by-table instructions
Data quality checklists
List of CSPR and EDFacts
coordinators by State
http://www.neglected-delinquent.org/nd/topics/index2.php?id=10
12
12
Collection Resource: NDTAC
NDTAC provides multiple products and services to
help State coordinators with data collection:
•
•
•
•
ND community calls
Annual CSPR Webinar
Data collection list
NDTAC presentations
– Better Data, Better Decisions by Bi Vuong
– Taking the Fast Lane to High-Quality Data by Stephanie
Lampron and Sarah Bardack
• Individual technical assistance
13
Reporting the Data
Federal data collection systems
Federal reporting timeline
Resources available for reporting
14
Reporting: Federal Reporting Systems,
CSPR
• CSPR is a data collection instrument administered
annually by ED’s Office of Elementary and
Secondary Education (OESE).
• The required measures for Title I, Part D, can be
found in CSPR section 2.4.
15
Reporting: Federal Reporting Systems,
EDFacts
• EDFacts is an ED initiative to collect, analyze,
report on, and promote the use of high-quality
performance data.
• Most of the Title I, Part D, data are now reported
through EDFacts. Eventually, all data will be
reported through the EDFacts’ online EDEN
Submission System (ESS).
16
Reporting: Federal Reporting Systems
Category/Area
CSPR Tables
EDFacts Specification
Files
Facility
Counts/Programming
S1=2.4.1.1 & 2.4.1.1.1
S2=2.4.2.1 & 2.4.2.1.1
Not included
Student
Count/Demographics
S1=2.4.1.2
S2=2.4.2.2
S1=C119 & C135
S2=C127 & C135
Transition Services
S1=2.4.1.3.1
S2=2.4.2.3.1
S1 & S2=C182
Academic/Vocational
Outcomes
S1=2.4.1.3.2
S2=2.4.2.3.2
In Program=C180
Exited Program=C181
Pre- and Posttesting in
Reading and Math
S1=2.4.1.6.1 & 2.4.1.6.2
S2=2.4.2.6.1 & 2.4.2.6.2
S1=C113 & C135
S2=C125 & C135
17
Reporting: Federal CSPR Reporting
Timeline
Establish reporting
timelines
18
Reporting Resource: EDFacts Coordinator
List
Identify who is responsible
for data submission in your
State and who has access to
EDEN
http://www2.ed.gov/about/inits/ed/edfacts/eden/ess/edfacts-coordinators.pdf
19
Reporting Resources: Federal Reporting
Systems
• CSPR
–
–
–
–
CSPR forms from ED
State CSPR coordinators
NDTAC’s CSPR guide, collection list, and annual CSPR Webinar
NDTAC’s data team individualized technical assistance about data
quality during CSPR collection
• EDFacts
– File specifications
– State EDFacts coordinators
– Partner Support Center (PSC)
• Phone: 877-HLP-EDEN
• Email: [email protected]
20
Evaluation
Federal use of data
State use of data
21
Evaluation: Federal Use of Data
• Program evaluation and performance
assessments
– Government Performance Results Act of 1993 (GPRA)
Performance Reports
– Federal Budget Requests to Congress
– In response to Congressional Requests (e.g.,
Elementary and Secondary Education Act
reauthorization)
• Public dissemination
Understand how ED
will use the CSPR
data
22
Public Dissemination of Data
ED Data Express
http://www.eddataexpress.ed.gov/
23
Public Dissemination of Data
National and State Fast Facts
http://data.neglected-delinquent.org/index.php?id=01
24
Public Dissemination of Data
NDTAC Annual Reports
http://www.neglected-delinquent.org/nd/topics/index2.php?id=12
25
Evaluation: State Use of Data
• States must use data to improve Title I, Part D,
programs, as directed by Subpart 3.
• Effective uses of data at the State level for
program improvement may include:
– Program evaluation
– Decision making
Use data for program quality
improvement
– Goal setting
• Data use at the subgrantee level also supports
program improvement.
26
Evaluation: State Use of Data
27
Evaluation: State Use of Data
Suggested areas for program improvement:
• Federal performance measures
– High school diploma/GED
– High school course credits
– Math and reading
– Cost per diploma
• State priorities
• Specific issues related to student population
28
Evaluation Resources: State Use of Data
• NDTAC publications
– Title I, Part D, Program Administration Planning Toolkit
 Application Review and Needs Assessment tools
– State and local Report Card resources
• Data and decision-making presentations
– Better Data, Better Decisions
– Making the Most of Your Data: Strategies for Evaluating
Your Program
– Using Pre- and Posttesting To Improve Programming and
Student Achievement
– Using Data To Identify and Promote the Implementation of
Effective Programming for Children and Youth Who Are AtRisk
29
Beyond the Basics
30
Beyond the Basics
• Establish goals for Part D programs
– Using the data, establish long- and short-term goals for
programs
• Expand upon the data collected
• Use all available data
– Use access you may have to additional data from these
programs (i.e., student-level data)
• Share the data
– Use data collected to promote programs
• Use data in program administration
– Needs assessment, subgrantee monitoring, application
reviews
31