Transcript Kentucky GSEG Early Childhood Outcomes Initiative

Kentucky GSEG
General Supervision Enhancement Grant
© Caroline Gooden, Kentucky Early Childhood Data System, 2007




The GSEG Outcomes and Process
Overview of Continuous Assessment Guide
and implications for First Steps
How KEDS is working for Preschool
Behind the Scenes – Ensuring valid and
reliable data
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Two Major Goals

Pilot the Standards and Continuous Assessment
Guide to Determine….
–
–
–

how well CBA address the standards and outcomes
how well children with disabilities fit into the framework
supports needed to implement in programs state-wide
Pilot the 3 National Early Childhood Outcomes for
Part C and 619 (Preschool) of IDEA
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Continuous Assessment Guide

Diagnostic section designed to support
Evaluation activities as outlined in IDEA with
respect to eligibility
–
Norm-referenced standardized instruments (908
KAR 2:130 & 635 of IDEA)
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Continuous Assessment Guide

Classroom/Instructional section designed to support
Assessment activities as outlined 908 KAR 2:130
with progress monitoring and program planning
–
–
Criterion referenced or curriculum based tools
“an ongoing process of observing a child's current
competencies (including knowledge, skills, dispositions and
attitudes) and using the information to help the child
develop further in the context of family and caregiving and
learning environments”
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Continuous Assessment Guide

Purpose of Classroom/Instructional
Assessments
1.
2.
3.
Identify the individualized needs of children to
inform intervention and instructional planning,
Develop individualized plans (e.g., IFSPs)
Inform families and other team members of child
developmental status.
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Why CR/CB over Norm-Referenced


Gather information for all the major developmental domains or
dimensions of development.
Tools were reviewed for their ability to:
–
–
–
–

Be developmentally and culturally valid;
Yield comprehensive and useful information
Allow for multiple methods from multiple sources without jeopardizing
reliability or validity
Connect to important learning represented in the Kentucky EC Standards,
Provide outcomes that match family goals and cultural preferences.
It is widely accepted that the tools included in the screening and
diagnostic assessment section of CAG do not provide
information sufficient for the purposes of classroom/instructional
program planning and this is supported in 908 KAR 2:130.
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Objectives of GSEG
1.
2.
3.
4.
Determine current use of the standards and
assessment guide
Help programs develop assessment
systems for data on outcomes
Help programs use assessment data for
child and program improvement
Document assessment process for
replication
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
GSEG Pilot Sites

Anderson Co. Preschool
–
–

Growing Together
–
–

Urban private early childhood center
~180 children, ages 0-5 years
2 Rural/urban early intervention providers
–

Rural public preschool center
~ 300 children ages 3 and 4
~20 children, ages 0-3
All serve children with & without disabilities
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Kentucky’s Approved
Assessment Tools

Preschool Child
Observation Record (COR)


Child Observation Record for
Infants & Toddlers (COR I-T)


Creative Curriculum


OUNCE Scale
Work Sampling System (WSS)
Brigance Inventory of Early
Development-II (IED-II)
Assessment, Evaluation, &
Programming System (AEPS)
Early Learning Accomplishment
Profile (E- LAP)






Learning Accomplishment
Profile–3 (LAP–3)
Transdisciplinary Play Based
Assessment (TPBA)
Hawaii Early Learning Profile
(HELP)
Carolina Curriculum for
Preschoolers with Special
Needs ( CCPSN)
Carolina Curriculum for
Infants and Toddlers with
Special Needs (CCITSN)
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Kentucky’s Approved
Assessment Tools








Preschool Child Observation
Record (COR)
Child Observation Record for
Infants & Toddlers (COR I-T)
Creative Curriculum
OUNCE Scale
Work Sampling System (WSS)
Brigance Inventory of Early
Development-II (IED-II)
Assessment, Evaluation, &
Programming System (AEPS)
Early Learning
Accomplishment Profile (ELAP)





Learning Accomplishment
Profile–3 (LAP–3)
Transdisciplinary Play Based
Assessment (TPBA)
Hawaii Early Learning Profile
(HELP)
Carolina Curriculum for
Preschoolers with Special
Needs ( CCPSN)
Carolina Curriculum for
Infants and Toddlers with
Special Needs (CCITSN)
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
GSEG Activities

Year 1 (2004-05)
–
–
–

Training on standards and assessment guide
Selection of assessment tools
Training on assessment tools by publishers
Year 2 (2005-06)
–
–
–
Collection of data on all children served by pilots
Training on assessment tools
Developing data platform for correlations to standards and
to OSEP child outcomes
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
What is KEDS?

Kentucky Early Childhood Data System

Provides a consistent platform for collection
of data from:
–
–
Diverse programs and providers across the state
Diverse numbers of approved assessment tools
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Why KEDS?

Supports improved instruction through
ongoing, continuous assessment
–


Gathering information in the context of everyday
routines and activities to obtain a representative
picture of children’s abilities and progress
Improves outcomes for children
Documents child outcomes for Office of
Special Education
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How Does Continuous Assessment
Work for Early Intervention Providers



Embedded with RBI
Supported by the
Consultative Model
Requires ongoing
communication among
team members
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How Does Continuous Assessment
Work for Early Intervention Providers



Providers continue to record observations,
field notes, gather “evidence” following a
home visit
Providers continue to communicate with
other members of the child’s team and family
members on progress (IFSP goals and
outcomes)
Providers record mastery of skills as
appropriate on assessment protocol.
© Caroline Gooden, Kentucky Early Childhood Data System, 2007

Example with Carolina
Curriculum
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How Does Continuous Assessment
Work for Early Intervention Providers

Providers record mastery of skills as appropriate on
assessment protocol.
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How KEDS was Designed for
Preschool

Ongoing meetings with OSEP, ECO and
KDE staff (accountability and early
childhood) to determine:
–
–
–
–
how the system should integrate with state data
system
who should enter data and at what level
what formats should be used by those in the field
how training and support will be provided for
assessments and data transmission
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How KEDS works for Preschool
Log In Accounts created
at two levels
 Preschool
Coordinators have
access to all children
in their program
 Teachers have
access to children on
their caseload
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How KEDS works for Preschool


Child level data is
imported into the
system from KDE
STI – child tracking
system
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How KEDS works for Preschool

Preschool
Coordinators
establish the
teacher
accounts and
assign children
to the teacher
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How KEDS Works for Preschool


Teachers conduct quality assessments on
each student:
Teachers tally data twice a year
–
–

November
May
Coordinators monitor teacher assessments
and data entry for accuracy
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How KEDS Works for Preschool

Teachers
verify
imported
data and
complete
additional
demographic
data on each
child
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Verification of Data










Child Unique Identifier Number
Date of Birth
Name
Gender
Ethnicity
IEP
Enrollment Date
Exit Date
District
School
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Additional Data

Does child have an IEP
–

Child Status
–
–
–

Developmental Delay, Speech/Language, or Severe
at-risk eligible (up to 150 % of poverty)
over income
LEP
Assessment used with child and format of the
assessment (on-line, diskette, or paper-and-pencil)
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Assessment Recording Formats



Online systems - direct administrative
access is provided to UK
Diskette or CD systems - electronic
submission of information via Keds Online
Paper-and-pencil versions
–
–
Publisher permission to establish data entry
system
Transmit data via Excel via Keds Online
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Sample Paper – Pencil Entry System
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
What Happens Behind the Scenes Methodology


Align individual assessment items to KY
standards and benchmarks then to OSEP
Outcomes
Activity
–
–
Conceptual framework
4 phase validity process
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
KY’s Conceptual Framework
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Benefits of the System




Allows flexibility for providers to choose a tool that
best meets the needs of individual children (with
state guidance)
Puts the focus on implementing high quality
assessments that provide immediate data to support
intervention
Immediately responsive to changes in required
outcome reporting at the federal level
Allows for reporting across multiple programs
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How Does
KEDS
Work?
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
4-Phase Validation Process

Expert Panel
–
–



General cross walk of all CB assessment tools to benchmarks and
outcomes
Detailed cross walks of all assessment items to benchmarks for
preschool tools
Statistical analysis of child data on benchmarks, standards and
OSEP outcomes (Correlational and Factor Analysis) for three
pilot assessments
Concurrent validity studies of 2 assessments using BDI-II
Teacher/Provider ratings and rankings of child progress on
standards, benchmarks and OSEP outcomes
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Sample Crosswalk
Arts & Humanities Standard 1: Participates and shows interest in a variety of visual art, dance, music and drama
experiences
Benchmark 1.1: Develops skills in and appreciation of visual arts
Developmental
Continuum
1)Uses
a variety of
media and
materials for
exploration (e.g.
paint, glue, threedimensional
materials,
technology, etc.).
2)Uses a variety of
art forms,
elements and
materials for
representing
people, places,
and things in the
environment.
3)Observes and
responds to
artwork produced
by other
individuals and/or
cultures.
Example Behaviors
1. Kenny chooses to paint at the
easel three days in a row. He
paints with one color, covering
the entire page.
2. Sally uses finger paint to
make swirls on black paper.
3. Shavon uses glue and ribbon
on paper.
4. The teacher puts Q-tips at
the art table and Monica dips
them in paint to make spots on
paper.
5. Ben uses a computer program
to create a picture then glues
on a tree-twigs picture frame
after printing the picture.
1. Olivia uses markers to draw
eyes, nose, and a mouth on a
paper plate.
2. Jarrad paints a picture of his
dog at the easel.
3. Maggie sticks leaves she has
gathered outside to a tree trunk
shaped from play-dough.
4. Trina builds her house with
Popsicle sticks and glue.
COR
I.1
J.1, 2
l.2-5
J.3-5
X.2
None
CC
WSS
LAP-3
None
37. F
through
III
None
NOTE:
FOR ALL
WSS
ITEMS,
ALL
ITEMS
REFER
TO 3’S
AND 4’S
UNLESS
OTHERW
ISE
NOTED
VI. A.3
None
VI. B.1
FM 4,
9, 10,
16,
19,
25,
30,
32,
35,
39;
PW 1,
2, 6,
8, 13,
16, 17
FM 34
PW
14, 29
None
AEPS
None
FM B
2.1
None
Brigan
ce
None
C-3.6,
3.7
None
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
CCPS
N
Help
Statistical Validation of Items

Data reduction
–

Establishing reliability
–

Are the scales measuring the content of the benchmark with
repeated use?
Ensuring internal consistency
–

Based on crosswalk analysis, single items pulled together to
create scales, a confirmatory factor analysis conducted to
ensure items are measuring the same dimension
Do the scales from the crosswalk “hang” together?
Predictive validity
–
Regression analysis
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Statistical Validation of Items

Conduct sample tests*
–
–

Measures of central tendency
Outlier analysis
Test individual assessment tools
–
–
–
Measures of central tendency
Outliers within tools
Investigate age bands - If no age bands are
available for individual assessments, investigate
continuum
* No group norms are available for B-3 tool selection, therefore in depth analysis of
the sample
B-3Childhood
will be the
main
focus2007
© Caroline Gooden,
Kentuckyfor
Early
Data
System,
Statistical Validation of Items

Convert individual assessment scores to common
metric
–
Allows for comparison between assessments that was not
possible before conversion

–
–
Conversion formula
x- µ
Z=σ
Conversion based solely on the individual assessment tool,
and once converted, can be compared to any other z score
Investigate the relationship of items and scales to KY
benchmarks
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Statistical Validation of Items


Align KY benchmarks to OSEP outcomes
Report entry data on Outcomes based on
OSEP’s five category reporting system
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
Final Conversations and Questions



Have the funding to support the system
design for First Steps and other CHFS early
childhood programs
Need to complete the validation work with 0
– 3 assessment tools
Need to identify specific program issues and
considerations for implementation,
specifically….
© Caroline Gooden, Kentucky Early Childhood Data System, 2007
How KEDS can be designed to support
Early Intervention

Continued meetings and discussions with
OSEP, ECO and EI staff to determine:
1.
2.
3.
4.
5.
How the system should integrate with state data
system – specifically CBIS
Who should enter data and at what level
What formats should be used by providers
How training and support will be provided –
have trainers available for most assessments
Data transmission processes – have basic
systems in place
© Caroline Gooden, Kentucky Early Childhood Data System, 2007