Transcript Community Input Discussions: Measuring the Progress of Young Children in Massachusetts August 2009

Community Input Discussions:
Measuring the Progress of Young
Children in Massachusetts
August 2009
Statement of Intent
2

Massachusetts is in the early stages of
developing a statewide system to measure
developmental progress of its young children

EEC is engaging parents, providers, program
administrators, teachers, higher education
institutions, and policy makers to build a
responsive approach

This initiative is separate from (and would
not replace) developmental information that
programs gather about children to use for
curriculum planning and to individualize
instruction
Questions for Stakeholder Input
3

What are your hopes for measuring the
developmental progress of young children in
state?

How do you hope information will be used?

What are the most important things to
measure about school readiness?

What are some of the challenges you foresee
in moving forward with this effort?
Measuring School Readiness
Across the Country
4

Massachusetts is joining rapidly growing
trend to understand school readiness

29 states currently collect statewide data
on children’s progress

These efforts are often directly connected
to state-funded preschool efforts
Why Are Statewide Data
Needed?
5

To inform policy makers about the benefits
of and impact of investments in early
childhood education and care in
Massachusetts

To better understand school readiness
gaps(s) for subgroups of children

To inform statewide policy development

Data will not be used for high stakes
“testing” of young children or providers
Statewide Measures of School
Readiness vs. Child Assessments
and Screenings

Currently, many providers are already using a
developmental assessment or screening tool to inform
practice and individualize instruction

Providers are currently using a variety of assessment
measures

UPK grantees are required to use one of four assessment
systems
•
•
•
•

6
Creative Curriculum Developmental Continuum
Ages & Stages
High Scope Child Observation Record (COR)
Work Sampling System
Statewide measurement of school readiness is not
intended to be used to replace program-level
assessment practices
Statewide Measures of School
Readiness vs. Child Assessments
and Screenings

Different purposes



Different level of information about child


7
Statewide system: information about the success
of all children in Massachusetts
Program-level assessments: information for
parents and caregivers about individual children
Statewide system: measure a small number of
indicators of school readiness
Program-level assessments: comprehensive look at
child progress across all developmental domains
Statewide Measures of School
Readiness vs. Child Assessments
and Screenings

Different usefulness to providers
Statewide system: for broader policy purposes
 Program-level assessments: to help provider
support each child’s growth and development


Other differences

8
Statewide system: children will be anonymous
when data are reported
Design Options for Statewide
Measure: Key Issues to Consider
9

WHAT to measure about child development

WHO to measure

WHEN to measure

HOW will measures/information on children
be collected

WHAT ELSE to measure about home
environments, program characteristics, and
community context
Design Options for Statewide
Measure: Key Issues to Consider
10

WHAT: how broadly should we measure
children’s development – which domains are
most important?

WHO: all children and programs, or just a
sample?

WHEN: longitudinal data in preschool only (ages
3 and 4) or linking with school data (K and/or
grade 1)?

HOW: providers trained to administer
measures, outside evaluators, and/or other
informants?
Other State Approaches
11

All states look at development either at one point
in time or over time, usually using different
samples of children at each age

All states interested in measuring school
readiness skills, although the skills that they
measure vary

Types of measures used vary (standardized
assessment vs. developmental observations,
number of development areas measured)

Where system is ongoing, states involve
providers in data collection
Other States: Approaches Using
Providers as Data Collectors
12

MD: Ongoing assessment of all children at school
entry by kindergarten teachers, using Work
Sampling System

PA: Ongoing measures of all children three time
per year during preschool and kindergarten by
providers, using Work Sampling System

NC: Snapshot of school readiness information
using variety of measures from a sample of
principals, kindergarten teachers, parents, and
children just entering kindergarten, coordinated
by research firm
Key Issues Raised To Date
(based on six stakeholder meetings)
13

Engage parent and provider support for initiative to ensure
maximum participation
 Make sure purpose of initiative is clear

Ensure information collected about children does not reflect
negatively on families

Consider how to support providers and minimize burden if
they will be on the front lines of the measurement effort

Consider how to provide context for data on development of
vulnerable subgroups of children

Consider linking child assessment data to program quality
data to better understand linkages between quality and
child outcomes
Key Decisions Moving Forward:
WHAT to measure?


14
Narrow vs. broad measurement
 Tension between desire to measure “whole child”
and what is feasible to collect
Which skills/outcomes to measure
 Focus on outcomes that research tells us are
related to success in school such as:
• Academic skills in reading, writing, and/or math
• Social skills
• Cognitive and behavioral self-regulation
Key Decisions Moving Forward:
WHO to measure?
15

Could measure as broad a population as
possible or target more narrowly
 Might oversample particular subgroups of
interest

Effort will likely be limited to preschool
children in licensed and license-exempt
settings
 Probably not feasible to access children
who are in informal and/or unlicensed care
settings or who are not in out-of-home
care at all
Key Decisions Moving Forward:
HOW to measure?



16
Providers as assessors
 Advantages: cost, may help with getting parent permission,
providers learn about their children’s skills
 Disadvantages: concerns about bias if providers assess their
own children, need to train large number of providers, difficult
for providers to find time to conduct quiet standardized
assessment
Outside assessors
 Advantages: can be trained to reliability, no public concerns
about partiality, possibility of building on early childhood
education infrastructure in state to develop group of assessors
 Disadvantages: cost of conducting assessments, cost of training
Parents as assessors
 Advantages: builds parent buy-in, not expensive
 Disadvantages: public perception of bias, some concepts might
be hard to explain to parents, may be difficult to get parents to
return this information
Key Decisions Moving Forward:
Consider Integration/Overlap with
Other Assessment Efforts
UPK grantees using one of four
specified child assessment tools
 Many other programs are using either
one of these four tools or another
measure
 School district programs using Battelle
in spring of 3-year-old year with
children who will be evaluated for
special needs
 Other efforts?

17
Key Decision Points Moving
Forward: WHEN to measure?
18

Options for data collection schedule before school
entry
 Measure children once, at the end of preschool, to
describe school readiness of preschoolers
 Measure children twice, at the beginning and end of
preschool, to describe school readiness but also to see
whether children who start out below average are
closing the gap over time
 Measure children three times, once during the 3-year-old
year and fall and spring of the 4-year-old year. This
design provides more information on progress over time

Looking at same group of children over time vs.
different groups of children each time
 Although following the same children would allow us to
look at individual growth over time, this is a very
expensive undertaking
Questions for Stakeholder Input
19

What are your hopes for measuring the
developmental progress of young children in
state?

How do you hope information will be used?

What are the most important things to
measure about school readiness?

What are some of the challenges you foresee
in moving forward with this effort?