Transcript Between the Lions Preschool Literacy Project

Between the Lions
Preschool Literacy Project
Evaluation Results
2005-2006
Project Goal
To improve language development and
other pre-literacy skills among low-income
children in child care centers
Project Intervention
Leaders in Literacy training
On-site technical assistance
Between the Lions materials
Evaluation Questions
How do the classroom environments of teachers
who participate in the BTL intervention compare
with the classroom environments of teachers
who do not participate in the BTL intervention?
How do the literacy outcomes for children whose
teachers participate in the BTL intervention
compare with the literacy outcomes of children
whose teachers do not participate in the BTL
intervention?
Participants in 2005-2006
Fall 2005 Intervention
– 4 Child Care Centers
11 Classrooms
Fall 2005 Control
– 2 Child Care Centers
5 Classrooms
Spring 2006
Intervention
– 7 Child Care Centers
(3 from Fall Intervention,
1 from Fall Control)
9 Classrooms (All
different from Fall)
Spring 2006 Control
– 3 Child Care Centers
5 Classrooms (All
different from Fall)
Instruments
Classroom Environment Assessment
– Early Language & Literacy Classroom
Observation (ELLCO) Toolkit
Early Literacy Assessment
– Get Ready to Read Screening Tool –
Recommended for 4-year-olds
– Test of Early Reading Ability-Third Edition
(TERA-3) – Designed exclusively for ages 3 ½ to 6
ELLCO Toolkit
Literacy Environment Checklist: Used in
assessing the organization and contents of the
classroom.
Classroom Observation & Teacher Interview:
Used to gather objective ratings of the quality of
the language and literacy environment and
experiences in a given classroom.
Literacy Activities Rating Scale: Provides
summary information on the nature and duration
of literacy-related activities observed.
Get Ready to Read Screening Tool
Print knowledge – Assesses a child’s
understanding of books, printed letters,
and words.
Emergent writing – Assesses a child’s first
efforts to use print in a meaningful way.
Linguistic awareness – Assesses a child’s
understanding of how language works.
Test of Early Reading Ability,
3rd Edition (TERA-3)
Subscales – Normed average score of 10
– Alphabet: Measures children’s knowledge of the
alphabet and sound-letter correspondence.
– Conventions: Measures children’s familiarity with the
conventions of print.
– Meaning: Measures children’s ability to comprehend
the meaning of printed material.
Reading Quotient – Normed average score of
100 – Derived from the sum of the three
subscale scores
Assessment Timeline
Fall 2005 Intervention
and Control
– Pretests - August 2005
ELLCO, Get Ready to
Read, TERA-3
– Posttests – Jan. 2006
ELLCO, Get Ready to
Read, TERA-3
– Delayed Posttests –
June 2006
ELLCO, Get Ready to
Read, TERA-3
Spring 2006
Intervention and
Control
– Pretests – January
2006
ELLCO, Get Ready to
Read, TERA-3
– Posttests – June 2006
ELLCO, Get Ready to
Read, TERA-3
Number and Percent of TERA-3 Participants
Below and Within the Acceptable Age Range
Below range
Within range
Fall 2005 Testing
Spring 2006 Testing
Intervention
Control
Intervention
Control
Number Percent Number Percent Number Percent Number Percent
20
18.0%
6
15.0%
37
33.3%
15
33.3%
91
82.0%
34
85.0%
74
66.7%
30
66.7%
Explanation of Statistical
Terminology Used
N - Number of participants
Mean - Average
p - Level of significance (p < .05 indicates
less than 5% probability that the effect was
a chance occurrence; 95% probability that
the change resulted from the intervention)
– Significance is determined by statistical tests
(e.g., t-tests)
Fall 2005 Classroom
Initial Assessment Results
Scale
Intervention Group Change
(Pretest to Posttest)
Control Group Change
(Pretest to Posttest)
ELLCO Literacy
Gain (significant)
Environment Checklist
Gain (not significant)
ELLCO Classroom
Observation Scale
Gain (significant)
Loss (not significant)
ELLCO Literacy
Gain (significant)
Activities Rating Scale
Gain (not significant)
Fall 2005 Preschoolers’
Initial Assessment Results
Scale
Intervention Group Change
(Pretest to Posttest)
Control Group Change
(Pretest to Posttest)
TERA-3 Reading
Quotient
Gain (not significant)
Gain (significant)
TERA-3 Alphabet
Subscale
Gain (significant)
Gain (not significant)
TERA-3 Conventions
Subscale
Gain (significant)
Gain (significant)
TERA-3 Meaning
Subscale
Gain (not significant)
Gain (significant)
Get Ready to Read
Total Score
Gain (significant)
Gain (significant)
Fall 2005 Classroom
Delayed Posttest Results
Scale
Intervention Group Change
(Pretest to Posttest)
Control Group Change
(Pretest to Posttest)
ELLCO Literacy
Gain (significant)
Environment Checklist
Gain (not significant)
ELLCO Classroom
Observation Scale
Gain (significant)
Loss (not significant)
ELLCO Literacy
Gain (significant)
Activities Rating Scale
Gain (not significant)
Fall 2005 Preschoolers’
Delayed Posttest Results (June 2006)
Scale
Intervention Group Change
(Pretest to Posttest)
Control Group Change
(Pretest to Posttest)
TERA-3 Reading
Quotient
Gain (significant)
Gain (not significant)
TERA-3 Alphabet
Subscale
Gain (significant)
Loss (not significant)
TERA-3 Conventions
Subscale
Gain (significant)
Gain (not significant)
TERA-3 Meaning
Subscale
Gain (significant)
Gain (snot isignifucant)
Get Ready to Read
Total Score
Gain (significant)
Gain (significant)
Spring 2006 Classroom
Assessment Results
Scale
Intervention Group Change
(Pretest to Posttest)
Control Group Change
(Pretest to Posttest)
ELLCO Literacy
Gain (significant)
Environment Checklist
Gain (not significant)
ELLCO Classroom
Observation Scale
Gain (not significant)
Loss (not significant)
ELLCO Literacy
Gain (not significant)
Activities Rating Scale
Loss (not significant)
Spring 2006 Preschoolers’
Assessment Results
Scale
Intervention Group Change
(Pretest to Posttest)
Control Group Change
(Pretest to Posttest)
TERA-3 Reading
Quotient
Loss (not significant)
Loss (significant)
TERA-3 Alphabet
Subscale
Gain (not significant)
Loss (not significant)
TERA-3 Conventions
Subscale
Loss (not significant)
Loss (significant)
TERA-3 Meaning
Subscale
Loss (not significant)
Loss (not significant)
Get Ready to Read
Total Score
Gain (significant)
Loss (not significant)
Limitations of 2005-2006 Study
Brief intervention periods for both Fall and
Spring (4 to 5 months)
Use of assessment instruments that were
not appropriate for younger aged children
Notably smaller control group than
intervention group
High turnover of teachers and students
Recommendations for
2006-2007 Study
Longer intervention period
Documentation of intervention to ensure
fidelity to intended treatment
Consideration of revising the process of
selecting intervention and control groups
to match more closely in number