Transcript The Portal Assessment Design System

Some Remarks on Quantitative vs. Qualitative
Reasoning in Educational Research
Robert J. Mislevy
University of Maryland
March 25, 2008
Presented at the Interactive Symposium Session “Generalizing From Educational
Research: Beyond the Quantitative–Qualitative Opposition ” at the annual meeting
of the American Educational Research Association, New York, March 25, 2008.
March 25, 2008
University of Maryland
Slide 1
Main Points

There is no quantitative analysis without a qualitative
frame.

Analyses by Hans-Georg Gadamer and John Tukey
have more in common than might appear at first
blush.

A project that needs to be done
March 25, 2008
University of Maryland
Slide 2
There is no quantitative analysis
without a qualitative frame.

Quantitative models are overlaid on a substantive
model concerning which concepts, entities,
events, and relationships.
March 25, 2008
University of Maryland
Slide 3
There is no quantitative analysis
without a qualitative frame.
Entities and
relationships
Real-World Situation
March 25, 2008
Reconceived Real-World Situation
University of Maryland
Slide 4
There is no quantitative analysis
without a qualitative frame.
Representational
Form A
y=ax+b (y-b)/a=x
Entities and
relationships
Real-World Situation
March 25, 2008
Reconceived Real-World Situation
University of Maryland
Slide 5
There is no quantitative analysis
without a qualitative frame.

Different possible levels / aspects of quantitative
models:
o Nature of real-world counterpart (Joel Michell)
o Relationships among variables
o Probability model for knowledge of analyst

Can be involved in any combination in a given
study
o One source of confusion in ‘qualitative vs. quantitative’
March 25, 2008
University of Maryland
Slide 6
Analyses by Hans-Georg Gadamer and John Tukey
have more in common than appear at first blush.

Come to problem of interpretation with some
concepts / models based on past experience.

“Hermeneutic circle” of play between understanding
of parts through whole, re-understand parts, modify
whole, etc.

Mosteller & Tukey’s Data Analysis and Regression

Coherence  fit
March 25, 2008
University of Maryland
Slide 7
Analyses by Hans-Georg Gadamer and John Tukey
have more in common than appear at first blush.

A bad statistician takes data and its interpretations as given,
fits model as statistical exercise. Policy-makers can mistake
model for the world (Scott’s Seeing like a State).

A good statistician carries out the interpretative interplay,
understanding data points as interpretations of unique
situations. Problematizes that step.

A great statistician pursues the interplay wherever it leads,
often to new understandings, new theories, new ways of
interpreting situations on the ground (Tukey, Deming,
Cronbach).
March 25, 2008
University of Maryland
Slide 8
A project that needs to be done

Meta-framework for describing ed. research analyses
o a la Greeno’s (1983) “Conceptual entities,” in D. Gentner & A.
L. Stevens (Eds.), Mental models.
o Must accommodate wide range of kinds of research studies,
spanned in Michael & Kadriye’s volume.
o Must show qualitative frames – necessarily hierarchical (levels
of schooling; interpersonal interactions; within-person models)
o Must show places/levels/aspects where quantitative models
can be used

Not ‘qualitative vs. quantitative,’ but how various model
spaces are constructed and how they interpenetrate
variously for different kinds of research questions.
March 25, 2008
University of Maryland
Slide 9