Making Sense of Qualitative Data
Download
Report
Transcript Making Sense of Qualitative Data
Qualitative Data Analysis
Tim Winchell
Analytical Techniques for Public Service
The Evergreen State College
Winter 2011
“It wasn’t curiosity that killed the cat.
It was trying to make sense of all the data curiosity generated.”
-Halcolm
Qualitative Data
“Qualitative Data… have been gathered during the conduct of
interpretive or postpositivist research studies. They exist most often
as some sort of narrative.” Examples include:
Written text
Conversation, interview, or
consultative transcriptions
Focus group transcriptions
Field notes
Diaries
Legal transcripts
Newspaper clippings
Journal or magazine articles
Photographs
Maps
Illustrations
Paintings
Musical scores
Tape recordings
Films (McNabb, p. 368)
Advantages
Grounded in a specific context/situation
Real life events/ settings; lived experience
Deep layers of meaning; rich description filled with differing
perspectives, symbolism, metaphor, and meaning.
“Descriptions form the bedrock of all qualitative reporting.”
(Patton, p. 438)
“The devil is in the details.”
Difficulties
Labor intensive
Research Bias
Requires creativity
Cost of processing/coding
Conceptual sensitivity
Non- formulaic
(Polit & Beck, p. 570)
data
Small sample size, many
variables
Very limited generalizability
Credibility
One Synopsis of the Challenges
“The challenge of qualitative analyses lies in making sense of massive
amounts of data. This involves reducing the volume of raw information,
sifting trivia from significance, identifying significant patterns, and
constructing a framework for communicating the essence of what the data
reveal…
There are no formulas for determining significance. No ways exist of perfectly
replicating the researcher’s analytical thought processes. No straightforward tests
can be applied for reliability and validity.
In short, no absolute rules exist except perhaps this: Do your very best with your
full intellect to fairly represent the data and communicate what the data reveal
given the purpose of the study.” (Patton, p. 432-433)
Data Management
The complexity of the project drives the level of organization needed
Format field notes consistently
Index notes, so you can find documents easily
Make sure you can read them!
Have a sensible system for cross referencing your notes
Please remember to:
Maintain data confidentiality as much as possible
Secure your data when not in use
Maintain participant confidentiality
And… Another Data Back Up Reminder!
“Thomas Carlyle lent the only copy of his handwritten
manuscript on the history of the French Revolution, his master
work, to philosopher J. S. Mill, who lent it to Mrs. Taylor. Mrs.
Taylor’s illiterate housekeeper thought it was waste paper and
burned it. Carlyle behaved with nobility and stoicism, and
immediately set about rewriting the book. It was published in
1837 to critical acclaim and consolidated Carlyle’s reputation as
one of the foremost men of letters of his day. We’ll never know
how the acclaimed version compared with the original or what
else Carlyle might have written in the year lost after the
fireplace calamity.” (Patton, p. 441- emphasis added)
Your General Approach?
Grounded
Start data collection with few preconceived notions about what’s
going on….no pre-formed coding scheme)
….or Framed?
Specific events, behaviors you intend to look for, with coding scheme
already partially developed. Oftentimes use diagrams to explain ideas.
All analyses benefit from diagramming and concept mapping, as Babbie
discusses (p. 405).
Qualitative Analysis:
The General Process
Data Reduction
Coding
Data Display
Conclusion Drawing
These are not linear, but concurrent processes
The less “framed” and more grounded the process, the more they
are concurrent: constant comparison
Data Reduction
First, we transform data from field notes or transcriptions
Write up and/or transcribe field notes and print.
Which of the data are most useful?
“Developing some manageable classification system or coding scheme
is the first step of analysis.
Without classification there is chaos and confusion.
Content analysis, then, involves identifying, coding, categorizing,
classifying, and labeling the primary patterns in the data.” (Patton, p. 463)
Consider….
For extensive research projects, summarize interviews with a
brief cover sheet
Who, what, where, when, importance, summary of key contacts
Coding schemes…must match the complexity of the project
Use similar semantics
Identifying concepts, patterns, memos
What is Coding?
In short, codes are shorthand descriptors of:
Setting and context
Subjects’ perspectives, which could include their thinking about
people and objects
Processes, activities, and/or strategies
Relationships and social structures
Any preassigned coding schemes
(Bogdan & Biklen, 1992, p. 166-172, as quoted in Creswell, p. 193)
Creswell recommends analyzing data using codes readers would
expect to learn more about, find surprising, and address larger
theoretical issues in the literature. (p. 193)
Variations….
Start categorizing early…
Or …..
Dive deeper into the data and avoid making judgments too
early… make tentative observations about what might be
happening….
To further analyze what is happening:
Write memos to yourself
Use “concept mapping” (Babbie, p. 405)
Build preliminary typologies
Try to use outcome/ process matrices (Patton, p. 468-477)
Open Coding….One Approach
Start with a sample of the data
Read responses carefully…
Keep research questions in mind
Make rough categories of these descriptors that seem to
belong together and code them with a key word.
Utilize constant comparison- similarities and differences.
Work to saturation.
Farm to School Example
Why do local farmers participate in the local farm to school program?
Resp.1: It makes the most business sense to me….
Possible code: ‘business sense,’ busin.
Resp. 2:“It gives me great pride to think of my organic produce being consumed
locally by my family members, friends, and church members and their children.
Possible code: “service,” serve
Farm to School Example
Business Sense (Busin.)
Service (Serve)
1. Most business sense
1. Belief in organic produce
being consumed locally
1. Organic production for
nuclear family, friends, &
church members & their
children
2. Service to local community
5. Some contribution to local
school district (lower prices
received)
3. Reduces transport costs
3. Ability to hire more
4. Reduces environmental
impact- transport
6. Stability of local school
district market
Write ongoing memos and abstracts
Comprehending:
The Basic Goal of this Stage
Identify important phenomena
Identify broad themes
Document codes that emerge
Begin to speculate about what might be happening
Write ongoing memos and abstracts
Axial Coding
Explore the relationships between and among codes
Look for:
“Contexts
Causal Conditions
Phenomenon central ideas
Strategies for addressing the phenomenon
Intervening conditions
Action/ interactions
Consequences” (Gibbs video)
Develop subcategories, linked by a “paradigm.”
Paradigm includes conditions, actions/ interactions, and consequences
(Polit & Beck, p. 584)
Employee Self Care Example
How could agencies promote employee self care in their organizations?
Organizational Changes
(OrgCh.)
Employee Changes
(EmpCh.)
Policies
Health Education
Management Training
Supervisory Best
Practices
Employee Awareness
Initiatives
Medical Coverage
Incentives
Individual Health
Surveys/ Contracts/
Teaming
Employee Best Practices
Selective Coding
Identify core phenomenon
Develop story line around the core concept(s)
Compare and contrast the core concept(s) to other selective
coding categories (Gibbs video)
Findings are integrated and refined
Include diagrams (Polit & Beck, p. 584)
Data Display
Playing with typologies and displays is a part of the analysis
process
See Miles and Huberman, Qualitative Data Analysis
Make sense of the data by playing with visual means of
representing the patterns that are emerging from the analysis
Process and outcome flow charts/ matrices
“Interpretation, by definition
involves going beyond the descriptive data. Interpretation means attaching
significance to what was found,
making sense of findings,
offering explanations,
drawing conclusions,
extrapolating lessons,
making inferences,
considering meanings,
and otherwise imposing order on an unruly but surely
patterned world.” (Patton, p. 480)
Theorize: Cause and Effect?
Classic Conditions for Establishing Cause and Effect
Variables Covary
Covariance is not spurious
Logical time order
A lucid explanation is available
Or …clusters of phenomena, identify things that tend often to show
up together, even if the causal connection is not clear
Qualitative Analysis- Visually
Analysis of Medical Errors
“Figure 1 classifies the stage in the diagnostic testing process and the
transition points within and between stages at which errors can
occur, and presents representative occurrences that fall into each of
them.” (Harris, et al.)
Early Introduction of Soft Foods by
Young Mothers
Verification
Triangulate from multiple sources or methods
Use several researchers as a reliability check.
Use rich, thick description in order to provide for the shared experience
Clarify research bias up front
Look for disconfirming evidence
Spend prolonged time in the field to develop an in-depth understanding
Use peer debriefing
Use an external auditor to review findings (Creswell, p. 196)
Complete several case studies. (Yin, 2003)
Review finding with participants.
If it’s just you, double or triple check your data and conclusions
Standards
Be true to the data
Don’t get too carried away by particularly eloquent,
memorable, or “simple” respondents—this creates a cognitive
bias
Always check and recheck both the data and conclusions you
draw from it
Qualitative Validity
Traditional Criteria for Judging
Quantitative Research
Alternative Criteria for Judging
Qualitative Research
Internal validity
Credibility
External validity
Transferability
Reliability
Dependability
Objectivity
Confirmability (Trochim, 2006)
Drawing Conclusions
Summary of data and results of coding analysis
Patterns and themes
Clusters of similar findings?
Case comparisons
Powerful metaphors
Any data for which your theory can’t provide a reasonable
explanation?
Final Thoughts
Data Management and Analysis work hand in hand
Coding is technical work, which is improved upon with advanced
practice, study, and interpretation
Remember to consult additional resource materials
(Some are listed at the end of the PowerPoint)
Utilize the Internet judiciously
Qualitative data software resources are reviewed in many
publications and on-line
Workshop Case:
TESC Alumni Relations
Research Interest
Why do colleges and universities have alumni programs?
Research questions
What are TESC graduates’ perceptions of TESC’s alumni programs?
What kind of alumni program do they want?
How do they recall their experience as TESC students?
What connects them to the College?
What nourishes that connection?
What can AR do to improve those connections?
Workshop Methods/ Results
Overview
Draft questions; approval from Alumni Relations
Zoomerang online survey
1647 responses
One researcher
Pluses: clear conclusions, grounded in data
Minus: not validated by second researcher
Workshop Exercise
Code 2 or 3 pages of the data from the responses to the Alumni
survey question.
“What was the best part of your experience at Evergreen?”
Code individual responses
What are the most common codes?
What do these data tell you/us about these alumni ? About
Evergreen?
Resources
YouTube Search “qualitative research coding”
Graham R. Gibbs Qualitative Research Coding Series
Open Coding:
http://www.youtube.com/watch?v=gn7Pr8M_Gu8
http://www.youtube.com/watch?v=vi5B7Zo0_OE&fe
ature=related
http://www.youtube.com/watch?v=nEomYWkxcA&feature=related
http://www.youtube.com/watch?v=AwmDRh5l7ZE&
feature=related
Resources
YouTube Search “qualitative research coding”
Graham R. Gibbs Qualitative Research Coding Series
Axial Coding:
http://www.youtube.com/watch?v=s65aH6So_zY&feature=r
elated
Selective Coding:
http://www.youtube.com/watch?v=w9BMjO7WzmM&featur
e=related
Grounded Theory:
http://www.youtube.com/watch?v=4SZDTp3_New&feature
=related
http://www.youtube.com/watch?v=dbntk_xeLHA&feature=
related
Morgan, D. L. (1997). Focus Groups as Qualitative Research (2nd
Ed.). Sage Publications: Thousand Oaks, CA.
Software Resources
Computer Programs:
See Babbie, p. 406-416
Data analysis strategies for qualitative research- Research
Corner
http://findarticles.com/p/articles/mi_m0FSL/is_6_74/ai
_81218986/?tag=content;col1
Software for qualitative research
http://homepages.vub.ac.be/~ncarpent/soft/soft_softsites
.html
Software for qualitative research
http://www.audiencedialogue.net/soft-qual.html
References
Babbie, E. (2010). The Practice of Social Research (12th Ed.).
Wadsworth Publishing: Belmont, CA.
Creswell, J. W. (2003). Research Design: Qualitative,
Quantitative, and Mixed Methods Approached (2nd Ed.). Sage
Publications: Thousand Oaks, CA.
Harris, et al. Mixed Methods Analysis of Medical Error Event
Reports: A Report from the ASIPS Collaborative
http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=aps2&part=A
2024
McNabb, D. E. (2002) Research Methods in Public
Administration and Nonprofit Management: Quantitative
and Qualitative Approaches. M.E. Sharpe: Armonk, NY.
References II
Miles, M. B., & A.M. Huberman. (1994). Qualitative Data Analysis.
(2nd Ed.). Sage Publications: Thousand Oaks, CA.
Patton, M. Q. (2002). Qualitative Research & Evaluation Methods
(3rd Ed.). Sage Publications: Thousand Oaks, CA.
Polit, D. F., & Beck, C. T. (2004). Nursing Research: Principles and
Methods (7th Ed.). Lippincott Williams & Wilkins: New York, NY.
Trochim, William M. K. (2006). Research Methods Knowledge Base.
http://www.socialresearchmethods.net/kb/qualapp.php
Yin, R. K. (2003) Case Study Research (3rd Ed.). Sage Publications:
Thousand Oaks, CA.
Acknowledgements
Making Sense of Qualitative Data
TESC MPA Program ATPS Winter 2010
Geri/Gould/McBride