THE EVOLUTION OF THE CONNECTICUT KINDERGARTEN ENTRANCE INVENTORY Peter Behuniak University of Connecticut

AERA Presentation

 Understanding Students’ Skills at Kindergarten Entry: Findings from Connecticut  Jessica Goldstein, Ph.D., Melissa Eastwood, M.A., & Peter Behuniak, Ph.D.

 Presentation at the Annual Conference of the American Educational Research Association, April 2012, Vancouver, B.C.

The presentation

 Connecticut’s Kindergarten Entrance Inventory (KEI)  Validity research  Predictive studies of the KEI  Quantitative study of structure of teacher ratings  Lessons for the future

A mandated measure

The 2007 Legislation required that:

“(h) Within available appropriations, the Commissioner of Education shall, not later than October 1, 2007, develop and implement a state-wide developmentally

appropriate kindergarten assessment tool that measures

a child’s preparedness for kindergarten, but shall not be used as a measurement tool for program accountability pursuant to section 10-16s, as amended by the act.”

From mandate to policy

LEGISLATIVE MANDATE

Developmentally appropriate kindergarten assessment tool that measures a child’s preparedness for kindergarten

CSDE POLICY

A statewide snapshot of the skills and behaviors students demonstrate, based on teachers’ observations, at the beginning of the kindergarten year

Structure of the KEI

 Census measure  Administered annually in October  Ratings assigned on 6 domains  Domains are defined by 3-5 indicators each

D1: Language skills

 Participate in conversations  Retell information from a story read to him/her  Follow simple 2-step verbal directions  Speak using sentences of at least 5 words  Communicate feelings and needs  Listen attentively to a speaker

D2: Literacy skills

 Hold a book and turn pages from the front to the back  Understand that print conveys meaning  Explore books independently  Recognize printed letters, especially in their name and familiar printed words  Match/connect letters and sounds  Identify some initial sounds  Demonstrate emergent writing

D3: Numeracy skills

       Count to 10 Demonstrate one-to-one correspondence while counting (e.g., touches objects as he/she counts) Measure objects using a variety of everyday items Identify simple shapes such as circles, squares, rectangles, and triangles Identify patterns Sort and group objects by size, shape, function (use), or other attributes Understand sequence of events (e.g., before, after, yesterday, today, or tomorrow)

D4: Physical/motor skills

 Run, jump, or balance  Kick or throw a ball, climb stairs or dance  Write or draw using writing instruments (e.g., markers, chalk, pencils, etc.)  Perform tasks, such as completing puzzles, stringing beads, or cutting with scissors

D5: Creative/aesthetic skills

 Draw, paint, sculpt, or build to represent experiences  Participate in pretend play  Enjoy or participate in musical experiences (e.g., singing, clapping, drumming, or dancing)

D6: Personal/social skills

 Engage in self-selected activities  Interact with peers to play or work cooperatively  Use words to express own feelings or to identify conflicts  Seek peer or adult help to resolve a conflict  Follow classroom routines

Rating scale

Performance Level

1

Description

Students at this level demonstrate emerging skills in the specified domain and require a large degree of instructional support.

Rating scale

Performance Level

1 2

Description

Students at this level demonstrate emerging skills in the specified domain and require a large degree of instructional support.

Students at this level inconsistently demonstrate the skills in the specified domain and require some instructional support.

Rating scale

Performance Level

1 2 3

Description

Students at this level demonstrate emerging skills in the specified domain and require a large degree of instructional support.

Students at this level inconsistently demonstrate the skills in the specified domain and require some instructional support.

Students at this level consistently demonstrate the skills in the specified domain and require minimal instructional support.

What do the data look like?

2007 KEI Ratings

Domain

Language Literacy Numeracy Physical Creative Personal

N

37048 37048 37048 37048 37048 37048

Mean

2.11

2.01

2.10

2.31

2.31

2.21

SD

.77

.76

.74

.69

.69

.73

Frequency 1

24% 29% 23% 13% 13% 18%

Frequency 2

40% 42% 45% 43% 42% 43%

Frequency 3

35% 29% 33% 44% 45% 39%

Classifications of validity evidence

AERA, APA, NCME Standards for Educational & Psychological Testing (1999)

Classification

Test content Relations to other variables Internal structure

Question

Does the KEI content match the CT Curriculum Framework?

Are KEI ratings consistent with scores on similar assessments?

Do KEI data match our expectations for test functionality? (Quantitative analyses) Response processes How do teachers evaluate and judge students’ skills and behaviors?

Test consequences Are the intended benefits of the KEI being realized? Are there unintended consequences of the KEI?

Validity evidence

Based on test content

Validity evidence based on test content  Indicators were developed from  Connecticut Preschool Curriculum Framework  Connecticut Preschool Assessment Framework  Connecticut Curriculum Standards for Language Arts  Connecticut Curriculum Standards for Mathematics  Indicators were reviewed by  Preschool and kindergarten teachers  Representation from urban and suburban districts, special education, and educators of English language learners

Validity evidence

Based on relationships to other variables

Validity evidence based on relationships to other variables  Are ratings on the KEI in kindergarten associated with performance on the state’s summative assessment in third grade?

 Fall 2007 Kindergarten Entrance Inventory data  Spring 2010 Grade 3 Connecticut Mastery Test  2007 KEI Ratings / 2010 CMT Data Study  Matched sample of students statewide (n = 29845)

2007 KEI Ratings by 2010 Grade 3 CMT Reading Proficiency

KEI Domain

Language

KEI Rating n Proficient+ Reading Grade 3 Not Proficient Reading Grade 3

3 13048 88% 12% Literacy 3 10867 89% 11% Numeracy 3 12097 88% 12%

2007 KEI Ratings by 2010 Grade 3 CMT Reading Proficiency

KEI Domain

Language

KEI Rating

1

n

9055

Proficient+ Reading Grade 3

54%

Not Proficient Reading Grade 3

46% 56% 44% Literacy 1 10657 Numeracy 1 8444 52% 48%

2007 KEI Ratings by 2010 Grade 3 CMT Reading Proficiency

KEI Domain

Language Literacy Numeracy

KEI Rating

1 2 3 1 2 3 1 2 3

n

9055 14945 13048 10657 15524 10867 8444 16507 12097

Proficient+ Reading Grade 3

54% 75% 88% 56% 77% 89% 52% 75% 88%

Not Proficient Reading Grade 3

46% 25% 12% 44% 23% 11% 48% 25% 12%

Validity evidence based on relationships to other variables   Are ratings on the KEI at the start of kindergarten associated kindergarten retention?

  Fall 2008 Kindergarten Entrance Inventory data (n = 40,713) Fall 2009 dichotomous retention variable

4% of 2008 kindergarten students were retained in 2009

The type of student who is the most likely to be retained for a second year of kindergarten is  Young    Male Eligible for free or reduced lunch Has KEI Ratings of “1” on Language, Literacy, Numeracy, and Personal/Social domains

Validity evidence

Based on internal structure

Language skills

 Participate in conversations  Communicate feelings and needs  Speak using sentences of at least 5 words  Follow simple 2-step verbal directions  Listen attentively to a speaker  Retell information from a story read to him/her

A new structure for teacher ratings

Domain Sub-Domain Sub-Domain Indicator 1 Indicator 2 Indicator 1 Indicator 2

Re-conceptualized language domain

LANGUAGE Expressive Language

Indicator 1 Indicator 2

Re-telling Information from a story

Indicator 1 Indicator 2

Receptive Language

Indicator 1 Indicator 2

Language domain in detail

Expressive Language Show comfort in expressing feelings and needs Express feelings and needs with words Speak using sentences of at least 5 words Communicate personal needs Communicate academic needs Communicate with peers Respond to “who, what, when, where” questions about self Actively participate in conversations using reciprocal dialogue

Language domain in detail

Receptive Language Listen attentively in small groups Listen attentively in large groups Follow simple two-step verbal directions Actively participate in conversations using reciprocal dialogue

Language domain in detail

Re-tell Information from a story Answer questions about a story Retell parts of a story Retell a story in proper sequence

Validity evidence based on internal structure

READY TO LEARN

A CADEMIC R EADINESS S OCIAL R EADINESS R EADINESS FOR A CTIVITIES

L ITERACY N UMERACY L ANGUAGE L ANGUAGE P ERSONAL / S OCIAL C REATIVE P HYSICAL / M OTOR

Validity evidence based on test consequences  Fall 2010 survey study of K teachers (n = 1084)  Teachers believe the KEI is an appropriate representation of students’ skills at the start of the kindergarten year.  Teachers felt the rating scale was appropriate.

 Teachers reported that they had appropriate time and training to complete the Inventory.  Teachers were neutral about administrative support to complete the instrument.

Looking across the studies

What have we learned?

   Teachers can form accurate judgments of student knowledge and skills at the start of the kindergarten year. Teacher judgments at the start of the kindergarten year are related to academic performance in later grades.

 Higher ratings at the domain level are related to higher levels of proficiency in each of the domains covered by the CMT. Future development of the KEI  Ratings of simple, discrete skills are easier than general domains.

  More detailed indicators provide more information about students. KEI focused on specific discrete skills could be rated dichotomously.

Conclusion

 Questions  Discussion  Follow-up issues or thoughts: [email protected]