Transcript PLS-5 Research Studies

Preschool Language Scales-5
Assessing Children from Birth through 7
Nancy Castilleja,
MA CCC-SLP
September 9, 2011
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Course objectives
•
•
•
describe at least two principles identified by ASHA
as best practices in early language assessment
identify three key differences between PLS-4 and
PLS-5
describe two research studies conducted with the
PLS-5
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Agenda
•
•
•
•
ASHA: best practices in early language
assessment
PLS-5 vs. PLS-4
–Start points
–Age level placements
–Scoring
PLS-5 Research Studies
–Development
–Minimizing Bias
–Standardization
–Reliability & Validity
Q&A
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Roles and Responsibilities of SLPs
in Early Intervention
•
www.asha.org /policy
•
Four guiding principles
1. family-centered and culturally and linguistically
responsive
2. developmentally supportive and promote
participation in natural environments
3. comprehensive, coordinated, and team based
4. based on the highest quality evidence available
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Family-centered and culturally and
linguistically responsive
–
Caregiver participation in testing
–
Caregiver-selected and caregiver-identified social routines and
vocabulary tested for very young children
–
Extensive bias review and widespread testing with a diverse
sample has resulted in
○
Familiar home vocabulary and contexts
○
Dialectal, regional, and cultural variations to identify
responses that are accepted as correct
–
Home Communication Questionnaire
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Sample from the Home
Communication Questionnaire
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Developmentally supportive and
promote participation in natural
environments
•
Observation of naturally-occurring behaviors for
younger children
•
Credit given for spontaneous productions in and
outside the assessment room
•
Developmentally appropriate skills assessed
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Comprehensive, coordinated, and
team based
•
Provides a survey of language skills in the
areas of
– Social communication
– Semantics
– Morphology
– Syntax
– Articulation
•
Useful for arena assessment; can be
administered by professionals in child
development teams
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Based on the highest quality evidence
available
•
Current review of research for item development
•
Expert review
•
Current normative data
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
What is new?
•
Current norms for a wider age range: based on the 2008
update of the U.S. Census for children birth through 7:11
•
Streamlined administration, with new suggested start points
and test items that involve manipulatives grouped for
smoother transitions during testing
•
New test items in the areas of play, understanding false
beliefs, and literacy
•
New Growth Scale Values, Evidence-based scores you can
use to track progress
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Based on the performance of
children in standardization…
•
New ceiling rule
– Basal: 3 consecutive correct responses
– Ceiling: 6 (not 7) consecutive errors
– Because
PLS-5 tests a variety of skills, a child could
obtain one or more score points if you discontinue
testing earlier
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Score multiple test items with one
procedure
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
New test items
For ages 18-24 months
–additional items assessing play
•
For ages 3 through 4 years
–new items assessing book
handling and concept of
print
•
For ages 5 through 7 years
–new items testing
– Theory of Mind
– Integrated language skills such as use of synonyms
– Emergent literacy skills such as naming letters and
understanding prefixes
•
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
New item: Uses synonyms
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
New Demonstration Item for
Understands Size/Sequence Concepts
(smallest, biggest)
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
New Practice item:
Uses Possessive Pronouns: hers,
his
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Items integrating Theory of Mind
concepts
•
Some items requiring that the child make
inferences about another’s feelings or intentions
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Understanding false beliefs
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
What is different?
•
Revised test items, based on clinicians’ and reviewers’
feedback
•
Updated Articulation Screener with picture stimuli
•
Start points
•
Age level placements of certain developmental skills
•
Normative scores
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Revised Test Items
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Modified Articulation Screener
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Frequently Asked Questions
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Why is “responds to speaker by
smiling” now at age 6-8 months?
“Start Points” are not developmental age levels
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
“Start Points” are not
developmental age levels
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Why do so many of the PLS-5 test
items appear at later age levels than
PLS-4?
•
Some clinicians are confusing the Start Points with
Developmental Levels
•
Criterion change for item placement
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Why do so many of the PLS-5 test
items appear at later age levels than
PLS-4?
•
Some clinicians are confusing the Start Points with
Developmental Levels
•
Criterion change for item placement at developmental
levels
•
The items were modified in some way
– The task is different
– Sub-items are different
– The number of sub-items has changed
– The pass criterion has changed
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Uses past tense
Age 4:
7%
Age 4:6: 42%
Age 5:0: 22%
Age 5:6 58%
Age 7:
78%
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
PLS-5 age level placements do not
match placement from Brown’s studies
•
According to Owens (2008), the age of mastery for Brown’s
developmental milestones is based on the research of:
– Bellugi & Brown (1964)
– Brown (1973)
– Miller (1981)
•
Demographic characteristics of the sample in 2010
•
When you review performance of a high SES sample,
performance on PLS-5 tasks match Brown’s seminal
research
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Some children are scoring higher on
the PLS-5? I’m concerned that children
will no longer qualify for services.
•
PLS-4/PLS-5 correlational study
– N = 134
– Average change:
○ AC: 1 point lower than PLS-4
○ EC: 1.5 points lower than PLS-4
○ Total Language: 1 point lower
– Range of score change for individuals higher
•
Protocols we’ve reviewed after publication
• Scores sometimes higher, sometimes lower
• Performance differences
• Item criterion differences
• No change in eligibility unless a child qualified with a 77 on
PLS-4
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
On the EC scale, the child hardly
needs to talk at all at age 2:0!
•
At age 2:0, 64-71% of the normative sample
– Used words more than gestures to communicate
– Used words for a variety of pragmatic functions
– Used different word combinations
That means 29-36% of children did not…
By age 2:6 to 2:11, the majority of the children in
the sample met criterion for these behaviors
•
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Standardization Research
•
Over 1800 children were tested for
standardization and related reliability and validity
studies from December 2009 through August
2010
•
The standardization sample was collected by 189
clinicians in 42 states in the United States
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Technical Information
Demographic Information
Validity Studies
– Clinical studies
○ Ages 1-2:11 language delay study
○ Ages 3:0-7:11 language disorder
study
○ Sensitivity/specificity
○ Positive/negative predictive power
– Correlations with other assessments
○ PLS-4
○ CELF Preschool-2
– Reliability Studies
– Internal consistency: typical
○ .80-.97
– Internal consistency: disorder
○ .96 to .98
– Inter-rater reliability: .95 to .98
– Inter-scorer agreement: .91 to 1.0
– Case studies
– Autism
– Hearing Impairment
–
–
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
PLS-5 Sample: Race/Ethnicity
African American 14.2%
Asian 4%
Hispanic 23.9%
White 53.7%
Sample mirrors March 2008
Update of the U.S. Census:
Race/Ethnicity
African-American 14%
Asian 4%
Hispanic 23%
White 55%
Other 4%
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Sample mirrors March 2008
Update of the U.S. Census:
Parent Education
0-11 years 15%
High School diploma or GED
26%
1-3 years college or technical
school 28%
4 or more years of college 31%
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Sample mirrors March 2008 Update
of the U.S. Census: Region
PLS-5 Normative Sample: Region
Northeast 19.7%
Midwest 19.9%
South 36.4%
West 24%
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Where Children in the PLS-5
Sample Spend the Majority of their
Day
Child's Learning Environment
Home w/Family 56.4%
Home w/Sitter 2.9%
Day Care 16.4%
Preschool 6.7%
Kinder
Grades 1-2
Other/NR
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Are children with disabilities
included in the PLS-5 sample?
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
PLS-4/PLS-5 Correlations
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
CELF Preschool-2/PLS-5 Correlations
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Clinical Study: Developmental
Delay
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Clinical Study: Receptive-Expressive
Disorder
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Sensitivity/Specificity:
NPP/PPP with 50% Matched Sample
AC
EC
Total
Matched
Sample 50%
Matched
Sample 50%
Matched
Sample 50%
PPP
.83
.86
.81
NPP
.77
.85
.83
PPP
.93
.94
.92
NPP
.66
.75
.74
PPP
.96
.95
.94
NPP
.60
.63
.63
Cut Score SD
and Predictive
Power
-1 SD
-1.5 SD
-2 SD
Note. PPP is Positive Predictive Power and NPP is Negative
Predictive Power.
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Minimizing Bias
•
•
•
•
•
Current literature review
Clinician surveys
Research studies
– Pilot
– Tryout (includes bias oversample)
– Standardization
Statistical analyses of bias
Expert review
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Questions related to the kit
1.
Do I need to purchase the manipulatives or can I
collect my own?
2.
Why can’t we have a Picture Manual we can wipe
clean?
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Questions related to the kit
How do you get the bear and the pitcher to fit in the
box?
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
I can’t get the bear to sit!
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.
Questions, concerns, comments?
www.speechandlanguage.com
Copyright  2011 Pearson Education, Inc. or its affiliate(s). All rights reserved.