Transcript Applicability of Vocabulary Measures with African American and

Vocabulary Testing: Diagnose,
Track Growth, & Intervene
Kathleen T. Williams, PhD
[email protected]
Snapshot of Speaker
• Began professional life as a second grade teacher
in 1968
• Involved in test and instructional material
development since 1989
• Has been a speech pathologist, classroom
teacher, school psychologist, graduate &
undergraduate instructor
• Has authored tests for vocabulary, reading, and
mathematics
• Has a BS in speech pathology, MEd in elementary
education, MA in educational psychology, and PhD
in school psychology
Overview
• Importance of assessing vocabulary
• Conorming tests
• Understanding scores
–
–
–
–
Normative or derived scores
Statistical significance of score difference
Prevalence of score difference
Tracking growth
• Generating hypotheses
– Receptive > Expressive
– Expressive > Receptive
• Qualitative analysis for designing intervention
strategies
What is “Vocabulary”?
• Lexical Store
– Mental dictionary of known meanings of words and
idioms
– Some may be inaccurate or incomplete
• Recognition vocabulary
– For listening and reading
• Productive vocabulary
– For speaking and writing
• Not the number of words one can decode in print
• It is the concepts one can understand at some
level
Levels of Word Knowledge
• Unknown
– Totally unfamiliar with the word
– Never hear it
– Never saw it in print
• Personal example:
– carapace
– “under the cold carapace of her adult
disappointments”
– a protective, decorative, or disguising shell
Levels of Word Knowledge
• Acquainted
–
–
–
–
–
Some familiarity with the word
May have seen or heard the word
Do not clearly understand it’s meaning
May understand its meaning in only one context
Could not use it precisely in writing or speaking
• Personal example:
– hessian
– “The scullery maid backed away, her head down,
wiping her dirty hands on her hessian apron.”
– burlap
Levels of Word Knowledge
• Unknown
– Totally unfamiliar with the word
– Never hear it
– Never saw it in print
• Acquainted
–
–
–
–
–
Some familiarity with the word
May have seen or heard the word
Do not clearly understand it’s meaning
May understand its meaning in only one context
Could not use it precisely in writing or speaking
• Well known
– Can comprehend the word when reading
– Can comfortably use the word when speaking or writing
– Understand its multiple means and uses
The Importance of Vocabulary
• Testing vocabulary is part of most
preschool screening batteries and Reading
First K to 3 programs
• Without a basic vocabulary, the potential
for developing a reading problem is great
• Not all children come to school with the
basic vocabulary knowledge required to
understand the directions and instructions
of the classroom
Limited Vocabulary – Lasting
Consequences
• Hart & Risley (1995) demonstrated that
early differences noted in children entering
school remain static throughout their
education
• Many children with an underlying language
disability also demonstrate poorly
developed oral vocabularies
• Plays a vital role in the development of an
individual’s academic and career
opportunities
• Vocabulary knowledge is strongly related to overall
reading comprehension
–Correlations range from .85 to .90
• If a word is decoded and pronounced but the meaning
is not recognized, comprehension will be impaired
• A limited vocabulary represents a limited
understanding of concepts
• Well-developed vocabulary skills and wide background
knowledge help individuals comprehend more difficult
and complex material
• The more words you know, the more you
can read
• The more you read, the more words you
will learn
• For children starting school with a limited
vocabulary, “more” reading does not
result in “more” vocabulary (Stanovich,
1986)
PPVT-4 & EVT-2
• Conormed measures of receptive & expressive
vocabulary
• Ages 2 ½ through 90+
• PPVT-4 (Peabody Picture Vocabulary Test, Fourth
Edition, Dunn & Dunn, 2007)
– Plate of four pictures
– Examiner says a stimulus word
– Examinee points to one picture
• EVT-2 (Expressive Vocabulary Test, Second Edition,
Williams, 2007)
– Single picture
– Examiner asks a question
– Examinee says one word that is a label or a synonym
• Two parallel forms to each test
Conorming
• Standardized on the same normative
sample
– Same population
– Same point in time
– Robust and stable measures of the
relationship
• Correlation of EVT-2 to PPVT-4 is .82
• Shared variance is 67%
Understanding Scores
• Limitations of raw scores
• Normative or derived scores
• Interpreting standard scores,
percentiles, normal curve equivalents,
and stanines
• Limitations of Age or Grade Equivalents
• Monitor progress or track growth with
GSVs
• What to do with extreme scores
Limitations of Raw Scores
– Not comparable from one test to another
– Tests have different numbers and types of
items
• PPVT-4: 228 receptive items
• EVT-2: 190 expressive items
– May have similar but unique growth
curves and varying score distribution
– Within a test, the same raw score does
not tell the same story
Normative or Derived Scores
• Reference a examinee’s performance to a
single group of interest
• For PPVT-4 & EVT-2, you have a choice:
– The referenced group can be based on the
individual’s age (2 ½ through 90), or
– The reference group can be based on grade
and time of school year (fall or spring)
• Normative scores include standard scores,
percentile ranks, normal curve equivalents,
and stanines
When to Use Age-based Scores
• Test results will be used as part of an evaluation
that considers age-based norms as best practice,
such as pre-referral evaluation as required by IDEA
• An examinee is not of the typical age range for his
or her same-grade peers due to retention or other
factors
• Testing is being completed in an atypical
educational setting, such as a summer school
program
• The formal educational background of the
examinee has been sporadic or uneven for some
reason, such as illness or transience
Interpreting Standard Scores
• Indicates the distance of a examinee’s raw score from
average, taking into account the variability of scores among
examinees of that same age or grade
• Mean of 100 and a standard deviation of 15
– SS of 85 to 115 = 68 percent of the group
– SS below 85 or above 115 = ~16 percent
– SS below 70 or above 130 = ~2 to 3 percent
• Equal interval scale of measurement
– Can add, subtract, multiple, & divide
• Must get the same standard score next year to demonstrate
a year of growth
Interpreting Percentiles
• Percentile rank indicates the percentage of
examinees in the reference group who performed at
or below a specific examinee’s score
– Percentile of 86 means the examinee scored as
high or higher than 86 percent of the normative
sample
• Percentiles are an ordinal or rank-order scale of
measurement
– Not an equal-interval scale of measurement
– Can not add, subtract, multiply, or divide
Normal Curve Equivalents
• Referred to as NCEs
• Range from 1 to 99, with a mean of 50 and
a standard deviation of 21.06
• Are based on percentiles but have been
statistically converted to an equal-interval
scale of measurement
– Can be averaged
– Useful for reporting data
Interpreting Stanines
• Stanines is a contraction of “standard
nines.”
• Single digit scoring metric with a mean of 5
and a standard deviation of 2
• Not an equal-interval scale of measurement
• They provide a “band” interpretation rather
than a single-score cut-off.
• Look for stanines that represent different
sections of the normal distribution
Developmental Norms
• Grade equivalents (GE) or age
equivalents (AE) place an examinee’s
performance along a growth curve across
grade or age
• A GE or an AE is the grade or age at
which a particular raw score is the
average score
• GEs are expressed in tenths of a grade
• AEs are expressed by years and months
Limitations of Grade or Age Equivalents
• An GE or AE does not necessarily mean that the
examinee’s vocabulary knowledge is qualitatively the
same as that of the average person at that grade or
age
• Should not be interpreted as a functional equivalent
(different raw scores can indicate different strengths
and weaknesses)
– An examinee aged 12 with an AE of 9:6 may tend to know a
different set of words than the average 9-year-old
• Are not an equal-interval scale of measurement
– Place examinees along a development continuum,
which may not increase at regular intervals
• Should not be used for diagnostic or placement
decisions
Understanding Growth Scale Values
• Growth scale values (GSVs) are included for both
PPVT-4 & EVT-2
• Developed so progress could be followed over a period
of years on the same continuous scale
– Standard scores, percentiles, NCEs, and stanines place an
examinee’s vocabulary knowledge with a reference group
(based on all those of the same age or grade)
• The GSVs provide a measure of an examinee’s
vocabulary knowledge in reference to the entire range
of growth across all ages or grades
– As an individual’s vocabulary grows, the GSV will increase
• An equal-interval scale
– Can be added, subtracted, or averaged
– Can be compared over time for many purposes such as RTI or
program evaluation
– Allows tracking of individual or group progress
Tracking Growth - Example
Age RS
SS
GSV
4:4
33
80
112
5:5
49
82
126
6:4
60
82
135
7:4
73
86
145
8:4
86
84
154
What to Do with Extreme Scores
• Very low scores
– No interpretation of a raw score of 0 on EVT-2
• It is lower than a score of 1, but how much can not be
determined
– No interpretation of a raw score of 3 or lower on
PPVT-4
• Corresponds to a score an examinee would obtain
through random responding
• Perfect or near perfect scores
– Should be interpreted with caution
• Report SS of 160 but note limitation on record form
– Examinee’s vocabulary may not have been measured fully
by the test
Qualitative Analysis of Results
• Calculate the absolute value of the difference
between PPVT-4 and EVT-2 standard scores
• Determine a level of significance of the
difference (Table B.7 in EVT-2 Manual)
– NS, .15, .10, .05, or .01
– Example: If significant at .05, 95% probability did not
occur by chance
• Find the percentage of the normative sample
with the obtained difference value (Table B.8 in
EVT-2 Manual)
Example 1
• Age 7 years 1 month
• PPVT-4 SS of 78
• EVT-2 SS of 84
– 6-point difference, not significant
• Could have occurred by chance
– Prevalence of difference
• Occurred in more than 25% of the sample
• Both SSs reflect a below-average
performance
Example 2
• Age 8 years 8 month
• PPVT-4 SS of 82
• EVT-2 SS of 73
– 9-point difference, significant at the .10 level
• 90% probability did not happen by chance
– Prevalence of difference
• Occurred in over 25% of the sample
• Fairly common
• Both SSs reflect a below-average performance
Example 3
• Age 6 years 6 month
• PPVT-4 SS of 81
• EVT-2 SS of 101
– 20-point difference, significant at the .01 level
• 99% probability did not happen by chance
– Prevalence of difference
• Occurred in only 5% of the sample
• Fairly uncommon
• Receptive score below average but expressive
score average
Receptive > Expressive
• Good guesser; may have some
knowledge of the word and can rule out
one or two of the options
• Good lexical store, but a problem with
word retrieval
• Broad experiential background; knows
many labels
• Strong knowledge of the prevailing
culture; knows the current label
Expressive > Receptive
• Lack of automaticity of word recognition
skills
• Deep processing vs. superficial
• Limited experiential background; knows
only one label
• Different cultural or “world knowledge”
base; can say an acceptable word
• History of hearing impairment; has learned
a specific set of labels
•“The greatest mistake we can make, I
believe, is to treat comprehension as a
simple process.”
–Theory, Assessment, and Intervention in
Language Disorders: An Integrative
Approach, by Elizabeth Carrow-Woolfolk,
1988
Qualitative Analysis
• Home versus school vocabulary
• Vocabulary by part of speech
• Crossover vocabulary sampling
between the EVT-2 and PPTV-4
• Three-tier model
Home Versus School
• Home words
– Words of high or moderately high frequency
that could be acquired through common life
experiences
– Labels children hear in a home environment
rich in language experiences
• Frequently read to
• “Lots of talk”
Home Versus School
• School words
– Not common to home environments
– Included in instructional lessons about exotic
or foreign places
– Labels for less concrete objects (geometric
shapes)
– Learned by reading newspapers, magazines,
or other informational materials
Home Versus School
• Home
– Household objects & food items
– Common objects found outside the home
– Basic numbers & colors
– Common domesticated & wild animals
– Actions (verbs)
– Attributes (adjective & adverbs)
– Body parts
Home Versus School
• School
– Less common domesticated & wild animals
– Geometric shapes
– Musical instruments
– Names of workers or occupations
– Recreational items not found in all homes
– Items specific to urban or rural environments
– Scientific instruments & specialized tools
– Geographic references
Home Versus School
• Division is arbitrary
• Above average score
– Probably has extensive knowledge from both
environments
• Below average score
– Lack of English vocabulary from home
environment
– Not profiting from educational environment
Vocabulary by Grammatical Form
• Part of speech
– Usage of word within the test
– Noun, verb, or attribute (adjective or adverb)
• Most early items are nouns
– Focus on labeling
• Actions & attributes
– Verbs, adjectives, adverbs
– More abstract than nouns
– Classified as “Home” since typically used in
conversation
Crossover Sampling
• Receptive & expressive common words
– PPVT-4 Form A & EVT-2 Form B
• 57 possible items
– PPVT-4 Form B & EVT-2 Form A
• 59 possible items
• EVT-2 items are open ended
– Labeling items: PPVT-4 item listed as common to
EVT-2 item only if the PPVT-4 stimulus word was the
most frequent correct EVT-2 response (90%)
– Synonym items: PPVT-4 item listed as common to
EVT-2 item only if the PPVT-4 stimulus word was at
least 70% of the correct response to the EVT-2 item
Three-tier Model
Low-frequency
words; technical
words
High-frequency,
high-utility
Beck, I. L., McKeown,
M. G., & Kucan, L.
(2002). Bringing words
to life: Robust
vocabulary instruction.
New York: The Guilford
Press.
Known,
common
words
Tier 3
Tier 2
Tier 1
• According to Farber (1999), many high
school students develop reading skills
through a sixth-grade level but “haven’t
actually read enough to develop the
vocabulary or general knowledge that
more advanced reading requires” (p. 1)
– Children must have vocabulary to learn to
read and comprehend, but then they gain
vocabulary by continued reading
• Good readers, as described by Lyon
(1997), “bring strong vocabularies and
good syntactic and grammatical skills to
the reading comprehension process…”
(p.5)
• Poor readers
– Lack of a strategy to learn new vocabulary
– May exhibit an overdependence on context
– Will expend too much effort to read each word
– May use only partial knowledge to determine
meaning
• Provide language-rich activities involving
listening and speaking
–Read or tell stories
• Ask questions
• Ask for a different word
• Ask examinee to retell story
–Tell jokes or tall tales
• Why is it funny? Unusual? Odd?
• Retell changing a key word
• Interactive Word Walls
• Brainstorm words that meet criteria (self-selection)
• Provide contexts for meanings
• Make associations from own existing knowledge and
experiences
• Apply words to real situations
– Get children to think more metcognitively about their own
vocabulary and the depth of their understanding about
particular words
• Have I heard the word used this way?
• Have I seen the word used this way in print?
• Have I used the word in my writing?
• Build new words using prefixes
• If centennial means 100 years, what does
bicentennial mean?
• Compound words
• What are the two words in the compound word?
– wiretapping, scatterbrain, underline
• What do they mean separately?
• What do they mean when used together?
Questions?
[email protected]