Transcript Thoughts on the Cognitive Psychology and Educational Assessment chapter

Educational Measurement, 4th Ed.
Chapter 8: Cognitive Psychology
And Educational Assessment
Robert J. Mislevy
University of Maryland
April 12, 2007
April 12, 2007
University of Maryland
Slide 1
Outline
Highlights of the chapter
 Changes between the 3rd and 4th
editions
 Content that was sacrificed due to
space limitations
 Issues or advances that have
developed since the chapter was
authored

April 12, 2007
University of Maryland
Slide 2
Highlights of the chapter



Evidentiary-argument framework
Psychological perspectives as grounding of
assessment arguments
Themes from cognitive psychology
»
»
»
»


Information-processing
Knowledge representation
Expertise research
Interactionalist perspective
Probability-based reasoning and cog psych –
based assessment
Examples of measurement models for cog psych
arguments
April 12, 2007
University of Maryland
Slide 3
Changes between 3rd Ed. and 4th Ed.
3rd Edition: Snow & Lohman
 First appearance of a chapter on this
subject
 Emphasis on information processing &
ability testing
 Look to integration of cognitive theory,
task design, and statistical modeling

April 12, 2007
University of Maryland
Slide 4
Key passage in 3rd Ed.
Summary test scores, and factors based on them,
have often been though of as “signs” indicating
the presence of underlying, latent traits. …
An alternative interpretation of test scores as
samples of cognitive processes and contents … is
equally justifiable and could be theoretically more
useful.
The evidence from cognitive psychology suggests
that test performances are comprised of complex
assemblies of component information-processing
actions that are adapted to task requirements
during performance. (Snow & Lohman, p. 317)
April 12, 2007
University of Maryland
Slide 5
Key passage in 3rd Ed.
The implication is that sign-trait interpretations of
test scores and their intercorrelations are
superficial summaries at best. At worst, they
have misled scientists, and the public, into
thinking of fundamental, fixed entities, measured
in amounts.
Whatever their practical value as summaries, for
selection, classification, certification, or program
evaluation, the cognitive psychological view is
that such interpretations no longer suffice as
scientific explanations of aptitude and
achievement constructs. (ibid)
April 12, 2007
University of Maryland
Slide 6
New to 4th Edition

Integration within evidentiary argument
framework
» How to implement vision from 3rd Ed.

Extension to substantive fields
» Expertise research
» Knowledge representation
» Research in learning domains
April 12, 2007
University of Maryland
Slide 7
New to 4th Edition

Extension to situative/sociocultural
perspective
» Complex performances
» Interpretive connection
» Minor theme: Associationist perspective

More substantive examples
» Model-based reasoning in science testing
» Architecture, network troubleshooting
April 12, 2007
University of Maryland
Slide 8
Sacrificed due to space limitations

More on associationist perspective
» Cognitive theory of cultural models
» Kintsch’s CI theory of comprehension
» Andy Clark on associationist perspective

Deeper examples
» Language testing (practical problems, sociolinguistics,
psycholinguistics, interactionalism,)
» Reading (comprehension; struggling readers)

Roles of technology
» Efficient use demands better understanding
April 12, 2007
University of Maryland
Slide 9
Issues since chapter was authored
NCLB
 Appropriate for Ed Measurement?
 Yes, probably:
» Snow & Lohman quote
» Relationship between learning and assmt
» What can, and cannot, be learned from
drop-in-from-the-sky assessment
» The situated nature of cognition
April 12, 2007
University of Maryland
Slide 10