Transcript Great Expectations: Practice and Policy for Developing a Quality Outcomes Assessment

Great Expectations:
Practice and Policy for
Developing a Quality
Outcomes Assessment
System
A Tale of Two States
Measuring Child and Family Outcomes 2008
Meredith Cathcart
Anne Kuschner
Larry Edelman
Patty Salcedo
1
California’s Child Outcomes System:
Quality Assessment to Quality Programs
 Overview of California’s DRDP Assessment
System
 Policies and practices that support quality
assessment systems
- Demonstration of an online learning module
designed for new assessors
 Use of data to improve services and
outcomes
2
The Context of Diversity in California
Size
Ethnic
Geographic
Linguistic
Socio-economic
3
Desired Results Access Project
•Overall Leadership and Coordination of Assessment Efforts
•Instrument Research and Development
•Training and Technical Assistance in Assessment
•Data Collection and Analysis
•Support for Use of Data
SEEDS
Training and
Technical
Assistance for
ECSE
SEECAP
Statewide
ECSE
Conferences &
Leadership
Training
CPIN
DR Grantees
ECSE Support
for Regular
Child
Development
LEA Partners
for Research &
Training in
Assessment
California Department of Education/Special Education Division
• Overall Direction
• Monitoring and Enforcement
• Policy Development
• State and Federal Reporting
• Funding
• Inter- and Intra-agency Coordination
California Department of Education
Desired Results System Assessment Instruments
Child Development Division
(CDD)
Desired Results
Developmental Profile Revised (DRDP-R) for:
• Infant/toddler
• Preschool
• School-age
Special Education Division
(SED)
• Preschool Desired Results
Developmental Profile –
Revised for Children with
IEPs (PS DRDP-R)
• Desired Results
Developmental Profile
access (DRDP access)
5
Purposes of the DRDP Assessment
System
 Assure that preschool-age children with IEPs
and their families benefit from California’s
ECSE services and programs
 Provide information about children’s
development in relation to research and
state standards so that programs, services,
and supports are designed to maximize
children’s potential
 Establish a “common ruler” for describing
children’s progress across the state
6
Goals of the DRDP Assessment
System
 Improve results for children and families
 Provide families with information about child
progress and ways to support their children’s
learning
 Provide teachers with authentic assessment
information to inform program planning
 Provide program administrators, staff, and
stakeholders with data to improve overall program
design
 Enable California’s preschool special education
system to comply with federal reporting
requirements
7
Principles Guiding the Development
of the DRDP Instruments
 Desired Results are for all children
 Content of the instruments is based on child
development research
 Instruments utilize an authentic assessment model
and are based on observation in typical
environments
 Multiple sources of data should be gathered over
time
 Assessments are universally designed
 Instruments include adaptations for children with
disabilities
8
Basics of California’s System
 In addition to the SPP, California must meet IDEA
requirements for statewide assessment:




- Every 3,4,and 5 year old is assessed 2x per year
- Assessment windows cover 6 month intervals
The SELPA is the unit of accountability for reporting
results
IEP teams determine which DRDP instrument to use
and which adaptations should be in place
Data is reported using a statewide web-based
system, SEDRS
T/TA system supports fidelity at all levels
9
Instruments
Systems
Change
Collaboration
Family
Participation
Training
California
Goals
&
Federal
Reporting
Making
Meaning
of Data
Technical
Assistance
Reporting
Data
…and so on…
Instruments
Systems
Change
Collaboration
Family
Participation
Training
California
Goals
&
Federal
Reporting
Making
Meaning
of Data
Technical
Assistance
Reporting
Data
The DRDP Instruments for
Preschool Special Education
12
Navigating the DRDP Instruments
Larry Edelman,
Desired Results access Project
Navigating the
DRDP Instruments
The Desired Results access Project
Napa County Office of Education
www.draccess.org
Funded by the California Department of
Education, Special Education Division
2
Our first online module in learning to use the
DRDP instruments
13
DRDP Adaptations
1. Augmentative or Alternative
Communication System
2. Alternative Mode for Written Language
3. Visual Support
4. Assistive Equipment or Device
5. Functional Positioning
6. Sensory Support
7. Alternative Response Mode
14
Training, Technical Assistance, and
Professional Development
16
Training Facilitates Quality
Assessment Practices
 Teachers understand the requirements and
responsibilities of the DRDP Assessment
System
 Teachers feel confident and competent
using the DRDP assessment tools
 The quality of data collected increases with
fidelity to the use of the instrument
17
What Worked for the First 2 Years of
Implementation
 Creating a phase-in of the system over several
years
 Creating a training cadre drawn from highly
qualified, experienced statewide trainers
- Built on collaboration between ECSE T/TA contractors,
SELPAs, and CDE
 Systematically mentoring less experienced trainers
 Conducting a mandatory Train-the-Trainer institute
 Piloting the training sessions
 Training trainers on the use of technology
 Insisting on minimum standards for training facilities
18
Current Training System Components
 Hold two statewide trainings annually for new
assessors
 Support and build capacity for local training events
 Provide online training modules for assessors and
data entry personnel
 Provide online guidance documents, teacher
resources, and discussion forum and listservs
 Disseminate updates at statewide and local
conferences and events
 Communicate with SELPA and link with CDE
19
A Brief Tour of www.draccess.org
20
Making Meaning of DRDP Data
21
What Data is Collected?
 Every preschool child with an IEP is
assessed on the ten DRDP Indicators twice
per year
 Current demographic information is gathered
on each child at each assessment period
 Raw data is submitted by assessors or
SELPAs to SEDRS
 Scale scores are generated by SEDRS and
reported back to SELPAs, district
administrators, and service providers
22
Primary Uses of Assessment
Information
 Serves multiple purposes:
- accountability
- monitoring of child progress
- program planning
 Provides immediate, useful feedback about
an individual child’s progress and status
 Assessment scores are easily understood by
staff and families, e.g.:
- plot a child’s developmental trajectory
- display status relative to typical development
23
The DRDP Contributes to:
 Formative assessment:
- information is gathered on an ongoing basis
- observation of child behavior is conducted in typical routines and
activities
- information is gathered from families, child care providers, and/or
related service providers
 Summative assessment
- data relative to the three OSEP child outcomes is aggregated by
CDE and sent to OSEP
- data for each SELPA is reported to the public according to OSEP
requirements
- data for programs is be provided through the use of DRDP scale
scores
24
The blue colored band
represents the typical
developmental range for
the child’s age
The black line marks the
place represented by the
child’s scale score in that
particular indicator
25
This report depicts the
change between two
assessment periods.
26
Group Measures into Indicators
27
Facilitated Process: Reflective Data
Analysis (pilot 2008-09)
 When data analysis is done within the context of the
team it yields useful information to programs and
adds to increased data quality.
 Build time to review and analyze data into team
meetings.
 Build a classroom and program summary
 Ask questions about the completeness and
accuracy of the data
 Ask questions about the implications of the results
for program design, services, and staff development
28
Some Questions about
Completeness and Accuracy of Data
 If children have a developmental age less than 3 years they
should NOT be assessed on the PS DRDP-R:
- Are IEP teams using an appropriate decision-making
process?
- What training is needed for IEP teams?
 Assessors should be able to rate 90% of Measures:
- Do teachers understand the developmental constructs
being measured by the DRDP?
- What are the barriers to observing collecting data?
 Most children with visual impairments should be assessed with
visual supports:
- What is the use of this adaptation in your program?
- Is more support needed for teachers?
29
DRDP
Results
Curriculum
Linkage
IEP goals
By Indicator
Specialty Curriculum
Functional
By OSEP
Outcome
General Curriculum
(commercial or locally
developed)
Academic
By
Developmental
Area
By Learning
Foundations/St
andards
Learning Foundations
/Standards
Child Development
Research
Age level
Standardsbased
Current Activities
31
Current Activities
 Continue activities to support quality professional
development
 Modify the DRDP access instrument to align with
Learning Foundations/preschool standards and
ongoing revisions to the PS DRDP-R
 Increase support to local communities on
professional development, data reporting, and data
analysis
 Expand activities to interpret results in useful ways
at local and state levels
 Create additional reports in the data collection and
analysis software (SEDRS) to assist teachers,
parents, and staff
32
Thank you!