Transcript Document

Working with
Qualitative Data
Christine Maidl Pribbenow
Wisconsin Center for Education Research
[email protected]
Session Outline
• General discussion about educational
research, assumptions and misconceptions
• Contrast educational research with research
in the sciences
• Define common qualitative analysis terms
• Provide example using ATLAS.ti–
qualitative analysis software program
• Code some text
Free Association…
DATA
QUALITATIVE
Qualitative Data:
Oxymoron or inherent tensions?
•
•
•
•
•
•
Hard vs. soft (mushy)
Rigor
Validity and reliability
Objective vs. subjective
Numbers vs. text
What is The Truth?
What are some of the assumptions
that you have about educational
research?
How are they helping or hindering
the development of your study?
Research in the sciences vs.
research in education
• “Hard” knowledge
• Produce findings that are
replicable
• Validated and accepted as
definitive (i.e., what we
know)
• Knowledge builds upon
itself– “skyscrapers of
knowledge”
• Oriented toward the
construction and
refinement of theory
• “Soft” knowledge
• Findings based in specific
contexts
• Difficult to replicate
• Cannot make causal
claims due to willful
human action
• Short-term effort of
intellectual accumulation–
“village huts”
• Oriented toward practical
application in specific
contexts
Strongly
Agree
Agree
Unsure
Disagree
Strongly
Disagree
Educational research is rigorous.
4 (31 %)
8 (62 %)
1 (8 %)
0 (0 %)
0 (0 %)
I have read at least ten articles
published in educational research
journals before attending this Institute.
11 (85 %)
2 (15 %)
0 (0 %)
0 (0 %)
0 (0 %)
Educational research is more difficult
than my scientific research.
1 (8 %)
2 (15 %)
6 (46 %)
3 (23 %)
1 (8 %)
I regularly collect qualitative data
in my classes for assessment purposes.
1 (8 %)
5 (38 %)
2 (15 %)
4 (31 %)
1 (8 %)
I need a control or comparison group
to conduct educational research.
2 (15 %)
3 (23 %)
1 (8 %)
5 (38 %)
2 (15 %)
Assessment data gleaned from students
(i.e., "self report") are valuable.
2 (15 %)
9 (69 %)
2 (15 %)
0 (0 %)
0 (0 %)
I have analyzed qualitative data in the
past.
1 (8 %)
3 (23 %)
1 (8 %)
4 (31 %)
4 (31 %)
Strongly
Agree
Agree
Unsure
Disagree
Strongly
Disagree
Qualitative data can meet "reliability"
standards.
2 (15 %)
5 (38 %)
6 (46 %)
0 (0 %)
0 (0 %)
Qualitative data can meet "validity"
standards.
2 (15 %)
5 (38 %)
6 (46 %)
0 (0 %)
0 (0 %)
If I collect learning assessment data from
my students and the analyzed results are
"not significant" it proves that students did
not learn what I intended.
0 (0 %)
0 (0 %)
1 (8 %)
5 (38 %)
7 (54 %)
If I conduct classroom research and the
results are "not significant",
the study was a waste of my time.
0 (0 %)
0 (0 %)
1 (8 %)
3 (23 %)
9 (69 %)
I need human subjects approval to
conduct and publish research
about my students.
7 (54 %)
1 (8 %)
5 (38 %)
0 (0 %)
0 (0 %)
I want to conduct research
in my classroom so that I can teach better.
11 (85 %)
2 (15 %)
0 (0 %)
0 (0 %)
0 (0 %)
I want to conduct research in my classroom
so that my students learn more or better.
13 (100 %)
0 (0 %)
0 (0 %)
0 (0 %)
0 (0 %)
What are some sources of
qualitative data?
•
•
•
•
•
•
•
Lab notebooks
Open-ended exam questions
Papers
Journal entries
On-line discussions
Email
Notes from observations
Qualitative Data Analysis
Qualitative analysis is the
“interplay between researchers and data.”
Researcher and analysis are
“inextricably linked.”
Qualitative Data Analysis
• Inductive process
– Grounded Theory
•
•
•
•
Unsure of what you’re looking for, what you’ll find
No assumptions
No literature review at the beginning
Constant comparative method
• Deductive process
– Theory driven
• Know the categories or themes using rubric, taxonomy
• Looking for confirming and disconfirming evidence
• Question and analysis informed by the literature
Example Research Questions
Why do faculty leave UW-Madison?
Do UW-Madison faculty leave
due to climate issues?
Definitions: Coding and Themes
• Coding process:
– Conceptualizing, reducing, elaborating and
relating text– words, phrases, sentences,
paragraphs.
• Building themes:
– Codes are categorized thematically to describe
or explain phenomenon.
Let’s Code #1
Read through the reflection paper written by
the student from an Ecology class and
highlight words, parts of sentences, and/or
whole sentences with some “code” attached
and identified to those sections.
What did you highlight?
Why?
Let’s Code #2
Read through this reflection paper and code
based on this question:
What were the student’s assumptions or
misconceptions before taking this course?
What did you highlight?
Why?
Let’s Code #3
Read through this reflection paper and code
based on this question:
What did the student learn in the course?
What did you highlight?
Why?
Can we say that the students
learned something in the course
using reflection papers?
Why or why not?
Ensuring “validity” and
“reliability” in your research
• Use mixed methods, multiple sources.
• Triangulate your data whenever possible.
• Ask others to review your design methodology,
observations, data, analysis, and interpretations
(e.g., inter-rater reliability).
• Rely on your study participants to “member
check” your findings.
• Note limitations of your study whenever possible.
Does the redesign of an ecology course to
include concept maps derived from
current journal articles help students to
gain a more current and realistic view of
ecological issues?
3 Sources
of Data
Concept maps of content
found in journal articles (Both)
Pre-post exam
of concepts (Quantitative)
Reflection Paper
(Qualitative)
Questions?
The plural of anecdote is data.
-Donna Shalala