Transcript What Works? Heide Spruck Wrigley

A Project of the U.S. Department of Education
Planning for a Successful Data System
•The webinar will begin at 2:00 PM Eastern
•You are prepared to view the presentation
•To listen by phone:
•Dial: 1-888-742-8686
•Conference code: 1975567
•Dial *0 for Raindance technical assistance
•To maximize view, go to ‘View: Full Screen’
Revised NRS Guidelines
Webinar Overview

Introduction


Planning an NRS Data System



2
David Hollender
Questions and Answers
A Longitudinal Data System


Larry Condelli
Steve Coleman
General Discussion
Revised NRS Guidelines
Planning An
NRS Data System
March 29, 2006
Dave’s “Top 3”
Reasons Not To Plan
1. Planning costs too much and takes too long.
We can skip it.
2. No need to plan. We’ll ask (fill in name of state) if
we can use their data system.
3. Our programmers are really smart.
They don’t need a plan.
4
Planning is Essential

Collect All Relevant Information

Meet All State & Local Needs

Assure Expertise & Infrastructure
Implementing an NRS data system without proper planning
is like teaching a class without a curriculum.
5
Steps to Success
1.
Analyze Requirements
2.
Prepare Requirements Document
3.
Buy / Build System
4.
Test
5.
Rollout
Over the years, software engineering practices have developed
that assure the success of new systems. These approaches
minimize the risk and help to manage the cost of new systems.
6
Analyze Requirements

Functionality

Usability

Technical
7
Analyzing Requirements
Functionality

Generate NRS Tables

Day to Day Program Needs

Support Decision Making

Promote Data Quality & Completeness

State Specific Needs
8
Analyzing Requirements
Usability

Who will operate system?

What will they need to do?

What Does “Easy to Use” Mean to them?
9
Analyzing Requirements
Technical

Timeliness

Security

Existing Infrastructure

Interoperability

Special Needs
10
Analyzing Requirements
Who Participates?

State Director

State Data Manger

State IT Department

Program Staff

Other State Staff
11
Prepare Requirements Document
For Whom ?
For What ?

You & Your Team

Review

Vendors/Developers

RFP

Acceptance Testing
12
Requirements Document Includes

Data Items

Required Functions

Reports

Technical Needs
13
Buy / Build System
Buy
Build

Base RFP on
requirements

Base Technical Spec
on requirements

Evaluate responses

Review design

Ask other states


Ask local staff
Build toward
benchmarks
14
Testing

Prepare Test Plan

Test Functionality

Realistic environment
15
Rollout

Configuration

Data Conversion

More Testing

Training

Technical Support
16
Steps to Success
1.
Analyze Requirements
2.
Prepare Requirements Document
3.
Buy / Build System
4.
Test
5.
Rollout
17
A Longitudinal Data System
What it can do for you
Stephen Coleman
American Institutes for Research
March 29, 2006
Planning for a Successful Data System
What is a Longitudinal Data System?



Allows analysis across PYs. Not limited to a
snapshot of adult education students.
Contains individual student records, which
allows much more flexibility in analysis and
tracking of students over time.
Able to produce reports which meet NRS
requirements.
Planning for a Successful Data System
Why a Longitudinal Data System?


Better understand the adult education
students in your state and therefore
better serve them.
Answer important questions about
adult education students in your state.
Planning for a Successful Data System
Questions it Can Answer


What is the relationship between intensity
of instruction, length of time in adult
education, and educational gain?
How many students who withdraw from
adult education programs re-enroll and
how long is the gap between withdraw
and re-enrollment?
Planning for a Successful Data System
Questions

Cont.
How long do students typically stay in
adult education?
What characteristics (e.g., age, EFL at
entry, program type) distinguish students
who stay 1 year or more from those who
stay for shorter periods?
Planning for a Successful Data System
Benefits

Provides a more accurate and dynamic
picture of adult education population.

Highlights changes across years.

Allows much more flexibility in analysis.
Planning for a Successful Data System
Benefits Cont.


Captures educational gain & other
characteristics regardless of when students
enter or leave or their length of stay.
Facilitates exploration of the relationship of
instructional variables to educational gain &
other outcomes.
Planning for a Successful Data System
Creating a Longitudinal Database




Create a relational database with a unique
ID for every student.
Collect all data you need and anticipate
needing.
Include students’ pre- and posttest scale
scores.
Create mechanisms to export data by a
range of dates.
Planning for a Successful Data System
Creating a Longitudinal…



Cont.
Include dates for time dependent events,
such as entering a program, testing, & goal
achievement.
Cleary and completely document the
variables in your database.
Clearly distinguish between missing data
and a negative response.
Planning for a Successful Data System
Summary



While may require additional initial
investment, has many benefits.
Provides more dynamic and accurate picture
of adult education students in your state.
Compatible with NRS reporting
requirements.
Planning for a Successful Data System
Additional Resources


NRS Tips Longitudinal Data Systems
Data Quality Campaign
(http://www.dataqualitycampaign.org)Creating a Longitudinal Data System and
Measuring What Matters: Creating a
Longitudinal Data System to Improve Student
Achievement.
Planning for a Successful Data System
Additional Resources

Cont.
ECS: A Policymaker’s Guide to the Value of
Longitudinal Student Data
(http://www.ecs.org/clearinghouse/40/21/4021.
htm).
A Better Student Data System for California
(http://wwwstat.stanford.edu/~rag/ed351/better_data.pdf)
Planning for a Successful Data System
General Discussion




To unmute your phone, dial *7
Say your name and the state you’re from
Ask a question or make a statement
Dial *6 to re-mute your phone
Planning for a Successful Data System
Webinar Evaluation




Your feedback is important. Please take our
short evaluation.
Select “File: Leave Meeting”
You will be automatically directed to the
evaluation.
Thank you!
Planning for a Successful Data System