Transcript Module 11 Slides Sue Flocke

Module 14
Overview
Qualitative Research
 Broad view of features
 Many traditions
 Contrasts with quantitative research
 Qualtitative method roadmap
 Data collection methods
 Standards of evaluation
Mixed Methods
• Approaches to mixing
• Examples
Research Methods
In the social sciences, there are 3 generic types of
research methods
 Quantitative methods
 Qualitative methods
 ‘Mixed methods’ blending quantitative and qualitative
Qualitative Research
 Focus on making sense of or interpreting social or human
phenomena: experiences, behaviors, organizational
functioning, social movements, interactional
relationships, descriptions of culture
 Phenomena are studied in their natural settings
 Investigators interpret phenomena in terms of the
meanings people bring to them
Denzin and Lincoln, 2000.
Qualitative work: Common features
 The researcher is the primary instrument for
data collection and analysis
 Involves fieldwork
 Attends to low inference descriptive data
Qualitative Methods - Traditions
Tradition
Purpose
Methods
Allied Discipline
Ethnography
Cultural
description
Intensive fieldwork
Anthropology
Phenomenology
Explore lived
experience
Analysis of cases to Philosophy
formulate meaning Psychology
Biography
Meaning of
experience as
recorded through
story
Interviews and
documents
exploring the life of
an individual
Literature
Literary Criticism
Grounded
Theory
Theory
development,
provide an
explanation
Constant
comparative
method
Sociology
Qualitative Research
 Attends to description of how vs. how many
 Is an inductive and open process: the investigator builds
concepts, hypotheses, and theories from analysis of the
data.
 The data collection and analysis process is iterative.
 The investigator returns to the data with new questions
and ideas to explore until there is a very deep
understanding of the phenomenon / issue.
Qualitative
Assumptions
 Reality is socially
constructed
 Primacy of subject
 *Variables are complex,
interwoven and difficult to
measure
 Relationships between
variables are generally
described as observed
patterns or cases
Quantitative
Assumptions
 Social facts have an
objective reality
 Primacy of method
 *Variables are measured
with existing tools
 Relationships between
variables can be assessed
using standard statistics
Qualitative
Purpose
 Contextualization
 Interpretation
 Understanding peoples’
perspectives
Researcher’s Role
 Personal involvement and
partiality
 Emphatic understanding
Design
 Flexible and emergent
Quantitative
Purpose
 Generalizable findings
 Prediction
 Causal explanations
Researcher’s Role
 Detachment and
impartiality
 Objective portrayal
Design
 a priori and inflexible
Qualitative
Purpose
 Contextualization
 Interpretation
 Understanding peoples’
perspectives
Researcher’s Role
 Personal involvement and
partiality
 Emphatic understanding
Design
 Flexible and emergent
Quantitative
Purpose
 Generalizable findings
 Prediction
 Causal explanations
Researcher’s Role
 Detachment and
impartiality
 Objective portrayal
Design
 a priori and inflexible
Qualitative
Quantitative
Approach
Approach
 End with hypotheses and
 Begins with hypotheses and







grounded theory
Emergence and portrayal
Naturalistic
Inductive
Searches for patterns
Seeks pluralism, complexity
Minor use of numerical indices
Thick description through
writing







theories
Manipulation and control
Experimentation
Deductive
Component analysis
Seeks consensus, the norm
Reduces all data to
numerical indices
Precise technical language,
numerical presentation
Questions?
The Circle of Inquiry
Identify
Knowledge
Gap
Search for
Existing
Information
Disseminate & Implement
Focus the
Study
Question
Analyze &
Interpret
Results
Collect
Data
Design the
Study
Adapted from: Nutting, PA, Stange, KC. Practice-based research: The opportunity to create a learning discipline.
In: The Textbook of Family Practice, 6th Edition. Rakel RE (ed.), W. B. Saunders Company, 2001.
Qualitative Data Collection Methods
 In-depth interviews
 Narratives, detailed responses to probing questions
 Direct observation
 Field notes containing descriptions of activities,
behaviors, actions, interactions, and processes
 Document analysis
 Written documents: records, memoranda,
correspondence, reports, diaries
Qualitative Methods Road Map
Sampling / Data collection
Describing
Organizing
Connecting
Corroborating / Legitimating
Crabtree & Miller, Doing
Qualitative Research 1999
2nd Ed.
Qualitative Methods Road Map
Sampling / Data collection
Describing
Organizing
Connecting
Representing the account
Corroborating / Legitimating
Qualitative Methods Road Map
Sampling / Data collection
Describing
Organizing
Connecting
Actively reflecting,
Specifically how what is
happening is influencing
& shaping the interpretive
process. Determining
next steps.
Representing the account
Corroborating / Legitimating
Qualitative Methods Road Map
Sampling / Data collection
Describing
Organizing
Connecting
3 general strategies:
- Template
- Editing
- Immersion /Crystalization
Corroborating / Legitimating
Qualitative Methods Road Map
Sampling / Data collection
Describing
Organizing
Connecting
Representing the account
Corroborating / Legitimating
Qualitative Methods Road Map
Gathering Process
Sampling / Data collection
Describing
Organizing
Connecting
Representing the account
Corroborating / Legitimating
Qualitative Methods Road Map
Sampling / Data collection
Describing
Organizing
Connecting
Corroborating / Legitimating
Analysis Process
Representing
the account
Qualitative Methods Road Map
Interpretive Process
Sampling / Data collection
Describing
Organizing
Connecting
Corroborating / Legitimating
Representing
the account
Qualitative Methods Road Map
Sampling / Data collection
Describing
Organizing
Connecting
Reflexivity
Representing the account
Corroborating / Legitimating
Questions?
Standards of Qualitative Research
God is in the details. R. Preston, 1994, p. 298
The devil is in the details. Popular adage.
Frankel, Standards of Qualitative Research. In Doing Qualitative Research, 2nd Ed.
Standards for Evaluation
Quantitative
 Internal Validity
Qualitative
 External Validity
 (Transferability)
 Reliability
 Confirmability, dependability
 Objectivity
 Engagement, reflexivity
 Credibility / Trustworthiness
Trustworthiness
 Ways to achieve trustworthiness:
 Member checks: recycling interpretation back
to the key informants
 Searching for disconfirming evidence
 Triangulation: multiple data sources and
multiple methods
 Thick description: a thorough description of the
context of the study
Confirmability
 Ways to address confirmability:
 Collection of data in ways that allow for audits



Audio recordings
Full transcripts of interviews
Collection of low inference observational data
 Engaging a team approach
 Independent auditors
Reflexivity
 Ways to address reflexivity:
 Document beliefs, framework, theories
underlying approach to the problem before
beginning the data collection.
 Actively journal reflections, possible biases/
limitations to ‘lens’ or ‘gaze’. Document how to
overcome limitations to more fully examine
phenomena.
 Engage other perspectives in team analysis.
 Report reflexivity in final product
Summary: What’s Involved in
Qualitative Inquiry
 Spend time in the field gathering data
 Engage in data analysis to reduce raw data to
themes or categories; complex & time-consuming
 Write extensively to show multiple perspectives in
order to substantiate findings
 Participate in a form of research that does not have
firm guidelines or specific procedures, and is
evolving and constantly changing
Source: Creswell JW. 1998. Qualitative inquiry and research design: choosing
among five traditions. Thousand Oaks, CA: Sage.
Qualitative/Quantitative
 Both approaches are highly respected
 When done well, both contribute equally to the
knowledge base
 When possible, researchers should blend the methods
 enriches the data to enable deeper understanding
 validates the conclusions from more than one
perspective
Summary of
Quantitative and Qualitative features
Feature
Quantitative
Qualitative
Role of researcher
Detached, objective
Engaged
Purpose
Test hypothesis
Describe;
Develop theory
Data collection methods A priori, structured
Flexible, iterative
Data analysis
Deductive, statistical Inductive, iterative
leading to more data
collection and
revision of question
Type of data
Numbers
Product
Status of hypothesis Rich description;
Theory development;
A book
Predominately Words
Mixed Methods
General consensus that no longer a
quantitative vs. qualitative methods – both
are necessary.
Mixed Methods
Attack the research problem with an arsenal
of methods that have non-overlapping
weaknesses in addition to complementary
strengths.
J Brewer & A Hunter Foundations of multimethod research: Synthesizing styles. Sage 2006.
page 4.
Mixed Methods
Scope of mixing methods
– within study
– within line of inquiry
– within broad topic area
Approaches to mixing methods
Sequential studies
Quantitative
Qualitative
Qualitative
Quantitative
Approaches to mixing methods
Mixing methods in the same study
 Dominant/ Less Dominant
Secondary method plays a small role
 Triangulation
Equal weight, both Quantitative and
Qualitative data collected & analyzed in a
complementary manner.
Examples of mixing
Data transformation – convert data of one type to
the other can be analyzed together.
Typology development – one type of data used
to develop a typology that is used to drive analysis
with other type of data.
Extreme case analysis – pursue data collection
or analysis of data of the other type with the intent of
refining the initial explanation for the extreme case.
Example line of inquiry
Observational Studies
DOPC
Intervention Studies
STEP-UP
Direct
Observation of
Primary Care
(NCI, RWJF:
1994-97)
Study To Enhance
Prevention by
Understanding
Practice
(NCI: 1997-2000)
P&CD
Prevention & Competing
Demands in Primary Care
(AHRQ: 1996-99)
IMPACT
Insights from Multimethod Practice
Assessment of Change over Time
(NCI: 2001-2004)
ULTRA
Using Learning Teams
for Reflective
Adaptation (NHLBI:
2002-07)
EPOCHS
Enhancing Practice Outcomes
through Community and
Healthcare Systems (2004-09)
Direct Observation of Primary Care
To accurately measure, using direct observation, the
level of preventive services delivered to eligible
patients, and to document the context of the
competing demands of the primary care medical
encounter
To determine the optimal non-observational
method…
To use a multimethod approach to identify and
explore which characteristics of the medical
encounter, physicians, patients and the office
environment are associated with delivery of
preventive services to eligible patients
Direct Observation of Primary Care
 Cross-sectional observation of 84 family
practices & 4454 patient visits to 138
physicians in Ohio
 Direct Observation
Davis Observation Code
Checklists





Medical Record Reviews
Patient Exit questionnaire
Billing Data
Practice Environment Checklist
Ethnographic Fieldnotes
Prevention & Competing Demands
PI – Crabtree
Aim: Understand primary care practice from the
ground up through intensive direct observation of
the practice environment and patient care to better
understand how these factors affect preventive
service delivery
Prevention & Competing Demands
 In-depth multimethod comparative case study of 18
family practices & 1,600 visits to 56 clinicians in
Nebraska
 Longer direct observation of practice environment
recorded in checklists and field notes (4-8 weeks of
observation)
 Direct observation of 30 encounters/clinician recorded
in checklists and field notes
 Chart audits of patients who were observed
 Interviews of all clinicians, most staff, some community
members
Study To Enhance Prevention by
Understanding Practice (STEP-UP)
 Randomized clinical trial of 80 family
practices in Ohio
 Multimethod assessment (MAP) of
values, structures, and processes
 Patient survey and medical record
review to assess preventive service
delivery at 6 month intervals
 Practice-individualized intervention
Doctor-patient communication study
Specifically this study aims to:
1. Characterize the way a ‘teachable moment’ arises
and is constructed within the patient-physician
interaction.
2. Identify factors that enhance or impede the use of
illness visits to provide health behavior advice.
3. Determine the effectiveness of a ‘teachable moment’
for increasing: patient recall of advice, motivation to
modify behavior, and change in health behavior.
Doctor-patient communication study
Graphic representation of mixing
Doctor-patient communication study
Outputs
Primary paper #1
Discovery of the teachable moment communication
process.
Primary paper #2
Quantitative testing the association of TM vs. other
kinds of advice with outcomes
Doctor-patient communication study
Embedded example
National Heart Lung & Blood guideline for assessment
and treatment of obesity.
- coding template of key guideline elements
- audio recordings, apply template
- findings from quantitative prompted qualitative
examination. (Unplanned)
Mixing methods
Expertise/training in methods you want to mix
- team members
- consultants
- cross training
- learning by doing*
Summary
The Power of Qualitative Data
 Stories are powerful and memorable.
 have a concrete, vivid, meaningful flavor that often proves far
more convincing and memorable to a reader – another
researcher, a policymaker, a practitioner
 Reports include explanations of processes in context and
derive explanations
 Enable investigators to address complex social issues
where little is understood. Methods push beyond initial
conceptions to generate or revise conceptual frameworks
Source: Miles MB, Huberman AM. 1994. Qualitative data analysis: an expanded sourcebook. 2nd
ed. Thousand Oaks, CA: Sage.
Making Decisions about which
Methods to Use
 What is the research question?
 What is the goal / purpose of the study?
 What kinds of information are needed to answer the Q?
 When is the information needed?
 What resources are available to conduct the study?
Given answers to the preceding questions, what methods
are appropriate?
Source: Patton MQ. 1990. Qualitative evaluation and research methods, 2nd
ed. Newbury Park, CA: Sage. p. 12
Questions?
Sue Flocke
[email protected]
Qualitative Analysis
 Affixing codes to interview data and field notes from





observations
Noting reflections in the margins
Sorting and sifting through these materials to identify
similar phrases, relationships, patterns, themes,
commonalities, & differences
Isolating patterns, processes, commonalities, &
differences and incorporating methods to further explore
them into the next wave of data collection
Gradually developing a small set of generalizations about
what consistently appears in the data
Confronting those generalizations with a formalized body
of knowledge in the form of constructs or theories
Themes of Qualitative Inquiry
Theme
Definition
Naturalistic Inquiry
Studying real-world situations as they unfold naturally; nonmanipulative; unobtrusive; lack of pre-determined constraints on
outcomes
Holistic perspective
The whole phenomenon is understood as a complex system that is
more than the sum of its parts; not reduced to a few discrete
variables and cause and effect relationships
Direct, close contact with people, situation, or phenomenon under
Personal contact and
study; researcher’s insights are important to inquiry and
insight
understanding
Source: Patton MQ. 1990. Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA:
Sage.
Themes of Qualitative Inquiry
Theme
Dynamic systems
Unique case orientation
Context sensitivity
Empathic neutrality
Design flexibility
Definition
Attention to process; assumes change is constant and
ongoing
Assumes each case is special and unique; captures details of
individual cases; cross-case analysis follows from individual
case studies
Places findings in a social, historical, and temporal context;
dubious of generalizations from other times and settings
Overriding objective is to understand the subject of
investigation in all its complexity rather than proving,
advocating, or advancing personal agendas
Open to adapting inquiry as understanding deepens and
situations change; avoids getting locked into rigid designs;
pursues new paths of discovery as they emerge
Source: Patton MQ. 1990. Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA:
Sage.