Transcript STANDARDS-BASED ACCOUNTABILITY SYSTEMS: DESIGN …

Learning from History:
Assessment & Improvement
of Student Learning
Lorrie A. Shepard
University of Colorado at Boulder
National Center for Research on Evaluation,
Standards, and Student Testing (CRESST)
CRESST Conference
UCLA
September 7-8, 2005
Previous Studies of
Historical Themes
• The role of testing in reform: 5 decades
50s tracking, 60s accountability, 70s minimum
competency, 80s school accountability, 90s
standards-based reform (Linn, 2000, ER)
• Behaviorist learning theory underlying criterionreferenced testing and teaching to the test
(Shepard, 1991)
• Historical conceptions of differentiated curriculum
and scientific measurement out of sync with current
conceptions of teaching and learning
(Shepard, 2000)
• Three cycles of test-based reform. Each corrected
narrowing effects of previous reforms but continued
the use of external tests to “incentivize” change.
(Shepard, 2002)
Past Efforts to Connect
Assessment and Instruction
E. F. Lindquist. (1951). Educational
Measurement.
• “the functions of educational measurement
are concerned…with the facilitation of
learning” (Cook, 1951).
• “educational measurement is conceived, not
as a process quite apart from instruction,
but rather as an integral part of it” (Tyler,
1951).
Our Measurement Forefathers
Made 3 Major Mistakes
• They trusted that objective formats were
sufficient to represent important learning
goals.
• They did not foresee the important
differences between day-to-day formative
assessment in classrooms and once-peryear formative program evaluation.
• They assumed (erroneously) that knowing
what students didn’t know would be enough
to know what to do about it.
Our assessment reform
efforts over the past 2
decades have been aimed
chiefly at correcting these
errors.
In the 1980s when research began to
show the negative effects of testdriven reforms…i.e. curriculum
distortion and test score inflation…
 Curriculum experts and measurement
specialists in the U.S. responded with
authentic assessments and
performance assessments.
 In Australia, the U.K., and New
Zealand, they responded with the
classroom processes of formative
assessment.
Our Measurement Forefathers
Made 3 Major Mistakes
• (Mistake # 1) They trusted that objective
formats were sufficient to represent
important learning goals.
• (Mistake # 2) They did not foresee the
important differences between day-to-day
formative assessment in classrooms and
once-per-year formative program
evaluation.
• They assumed (erroneously) that knowing
what students didn’t know would be enough
to know what to do about it.
© Wyoming Body Evidence Activities Consortium
© Wyoming Body Evidence Activities Consortium
Researchers in England, Australia,
New Zealand, Scotland, & Wales
opposed standardized tests by
focusing on formative assessment.
 (Crooks, 1988). Reviewed motivational and
cognitive effects of classroom assessment.
 (Sadler, 1989). Proposed a model of
formative assessment including feedback
and self-monitoring.
 (Assessment Reform Group, 1999).
Promoted the idea of “assessment for
learning.”
Paul Black & Dylan Wiliam (1998).
Assessment and Classroom Learning
 “Assessment becomes ‘formative
assessment’ when evidence is actually
used to adapt the teaching work to meet
student needs.”
 Formative assessment experiments
produce effect sizes of .40 - .70, larger
than found for most educational
interventions.
 Many studies show that improved
formative assessments help low achievers
most.
Knowing What Students Know
Pellegrino, Chudowsky, & Glaser. (2001). NRC.
Cognitive science findings on key aspects
of learning processes can be translated
into targeted features of formative
assessment:
• Accessing prior knowledge
• Strategic use of feedback
• Teaching and assessing for transfer
• Meta-cognitive benefits of self-assessment
Shepard (2000). “The Role of
Assessment in a Learning Culture.”
 “We have to change the social meaning of
evaluation.”
 Dynamic assessment and instructional
scaffolding in Vygotsky’s ZPD are essentially
the same processes.
 Developing an identity of mastery occurs as
learners participate in a community of
practice (Lave & Wenger, 1991).
 Self-assessment also increases students’
responsibility for their own learning and
makes the student-teacher relationship more
collaborative (Gipps, 1999).
Subject-matter
experts blended
content and process
reforms.
 Marie Clay (1985). Invented
assessment strategies
embedded in the acts of
reading.
 Yetta Goodman (1985).
Reintroduced the concept of
“kidwatching.”
 Teal, Hiebert, & Chittenden
(1987). Collected samples of
student work to gain insight
into children’s thinking and to
document progress over time.
NCTM Standards (1989) and
Everybody Counts (NRC, 1989).
 Learning mathematics as a process of inquiry
and sense making, not “mindless mimicry.”
 More extended, non-routine problems needed
to engage students and to assess
“mathematical power.”
 Classroom discourse became a focus to
provide students with the opportunity to
conjecture and explain their reasoning.
 Embedded, informal assessments –
observations, teacher questioning, and
journal writing – to gain insights into
students’ thinking.
 Curriculum-embedded assessments are much
more likely to produce coherent and
instructionally relevant assessment insights.
Most importantly they help teachers gain
qualitative insights about student
understandings to build on student
strengths.
 In contrast, formal benchmarking systems
mean administering earlier and earlier
versions of external tests at quarterly or
monthly intervals. The result is a long list of
discrete skill deficiencies requiring
inexperienced teachers to give a 1000 mini
lessons. These systems threaten to hijack
formative assessment reform efforts by
repeating mistakes 1 and 2.
Our Measurement Forefathers
Made 3 Major Mistakes
• They trusted that objective formats were
sufficient to represent important learning
goals.
• They did not foresee the important
differences between day-to-day formative
assessment in classrooms and once-peryear formative program evaluation.
• They assumed (erroneously) that knowing
what students didn’t know would be enough
to know what to do about it.
What about mistake # 3 as yet
unaddressed?
 Curriculum-embedded assessments
help if teachers have the training to
make qualitative judgments and to
build on existing knowledge.
 Curriculum-embedded or benchmark
assessments help if they are built to
uncover specific misconceptions.
 Learning progressions go further than
isolated assessment tasks because,
by definition they are tied to a model
of developing competence.
Beyond Curriculum-Embedded
Assessments, Learning Progressions
answer the question about what to do.
 An understanding of learning progressions or
learning trajectories is important for
monitoring and supporting learning and
development over time.
 How learning typically unfolds helps teachers
know “what next” and how to “back up”
(though we must also be aware of natural variations
and departures from the typical pattern).
 Learning progressions or “Progress Maps”
also provide an underlying model of learning
to coherently link classroom and large-scale
assessments (KWSK).
Geoff Masters
& Margaret
Forster (1996).
Developmental
Assessment
Progress maps
describe skills,
understandings, and
knowledge in the
sequence in which
they typically
develop: a picture of
what it means to
‘improve’ in an area
of learning.
North Carolina Children’s Writing Continuum
Strategies in Children's Spelling at Different Stages
Description
Example
Prominent Strategy
Prephonemic
Letters are used to write words but the sound-symbol
relationships are unrelated to target word.
"'C" for "hat"
Early phonemic
Some phonemes are represented by letters, typically
most salient phoneme(s) in a word.
"DR" for "Dear"
Phonetic
Attempts are made to represent most sounds in
words, often letter name that most closely resembles
sound.
"wns" for "once"
Simple Associations
Simple vowels and consonants are represented
correctly but complex patterns are not.
"bid" for "bird"
Strategic Extensions
With complex vowels and consonants, attempts
reflect complex English patterns, although not the
conventions of English.
"bote" for "boat"
Conventional
Hiebert and Raphael (1998)
Marja van den HeuvelPanhuizen (2001). In the
Netherlands, “learning-teaching
trajectories” are being developed to
provide pedagogical insights needed to
support the development of students’
thinking over time.
3 interwoven components:
• A learning trajectory describes the
learning process.
• A teaching trajectory links teaching
strategies with the learning process.
• A subject matter outline reflects the
core elements of the mathematics
curriculum that should be taught.