Transcript Mixed Methods Research

by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln, and Founding Co-Editor, Journal of
Mixed Methods Research
University of Pretoria, October 21, 2008
© Please do not duplicate or use these slides without the express permission of the author.
1
Topics
 Introductions
 Positioning myself
 A definition of mixed methods research
 Steps in the process of designing a mixed methods
study
2
Positioning myself
 View research as set of interactive components;
not always linear
 Focus on rigorous data collection and analysis
 Work as an applied research methodogist
 Trained in quantitative, self-trained in qualitative,
lst generation mixed methods writer
 Serve as a consultant on mixed methods on
projects
 Work on projects in an order not in proposal
format; then I reassemble into proper format
A few more thoughts about myself…
Published by Sage Publications, Pearson Education (Merrill Educ.)
A Definition of Mixed Methods
Research
How would you combine two types of data?
Qualitative
Text Data
 This is a sample of a text file of
words that might be collected
on interview transcripts,
observation fieldnotes, or
optically-scanned documents.
Quantitative Numeric Data
 23425231123423321
11534123144554121
43351423155221535
13153225132443124
2241554215
Framework for viewing perspectives on
mixed methods
Quantitative Data
Qualitative Data
Mixed Methods
Method
Paradigm
Perspective
Methodology
Use of mixed methods
in other designs
7
A mixed methods researcher…
 Collects both quantitative and qualitative data
 “Mixes” them
 “Mixes” them at the same time (concurrently) or one
after the other (sequentially)
 Emphasizes both equally or unequally
A definition
• Mixed methods research is both a method and methodology for
conducting research that involves collecting, analyzing, and
integrating quantitative and qualitative research in a single study or
a longitudinal program of inquiry.
• The purpose of this form of research is that both qualitative and
quantitative research, in combination, provide a better understanding
of a research problem or issue than either research approach alone.
9
Collecting both quantitative and
qualitative data
 Quantitative data
 Instruments
 Checklists
 Records
 Qualitative data
 Interviews
 Observations
 Documents
 Audio-visual materials
10
Quantitative and qualitative data
analysis
 Quantitative analysis
 Qualitative analysis
 Use statistical
 Use text and
analysis,
 For description
 For comparing
groups
 For relating
variables
images,
 For coding
 For theme
development
 For relating themes
11
Mixing or linking the data
Converge data:
Qual
Results
Quan
Connect data:
Qual
Quan
Results
Embed the data:
Quan data
Qual data
12
Typical situations in which mixed methods is
used…
 To compare results from quantitative and qualitative
research
 To use qualitative research to help explain quantitative
findings
 To explore using qualitative research and then to
generalize findings to a large population using
quantitative research
 To develop an instrument because none are available
or useful
 To augment an experiment with qualitative data
What is the reason for using mixed methods?
 The insufficient argument – either quantitative or qualitative may be





insufficient by itself
Multiple angles argument – quantitative and qualitative approaches
provide different “pictures”
The more-evidence-the-better argument – combined quantitative and
qualitative provides more evidence
Community of practice argument – mixed methods may be the
preferred approach within a scholarly community
Eager-to-learn argument – it is the latest methodology
“Its intuitive” argument – it mirrors “real life”
Designing a Mixed Methods Study










Preliminary considerations
Creating a title
Posing a general question
Listing the types of data collection and analysis
Making explicit your worldview
Identifying your research design
Drawing a figure of your design
Writing a purpose statement
Writing research questions
Completing a research plan
Preliminary considerations (before you begin
to design)
• Research problem
• Content – any topics
• Fit the problem to mixed methods (arguments)
•Access to both qualitative and quantitative data
•Background and resources
•Receptive audience
16
Create a working title
 Writing the title
 Short
 Topic
 Participants
 Include the words “Mixed methods”
 Neutral –neither quan or qual
17
Pose the general question to be answered
 Write it as a question
 Look to see how it is phrased
 Make sure that it is specific enough and focused (an
answerable question)
 Ask yourself, “when I end the study, what question
would like to have answered?”
18
List your types of data collection (a
review)
 Quantitative data
(closed-ended)
 Instruments
 Behavioral
checklists
 Records
 Qualitative data (open-ended)
 Interviews
 Observations
 Documents
 Audio-visual materials
19
Activity – List your sources of data
Quantitative Sources of
Data
Qualitative Sources of
Data
20
List your approach to data analysis (a
review)
 Quantitative analysis
 Qualitative analysis
 Use statistical
 Use text and
analysis,
 For description
 For comparing
groups
 For relating
variables
 Design-type
images,
 For coding
 For theme
development
 For relating themes
 Design-type
21
Determine your worldview…
Postpositivism
Determination
Reductionism
Empirical observation and
measurement
Theory verification
Constructivism
Understanding
Multiple participant meanings
Social and historical
construction
Theory generation
Advocacy/Participatory
Political
Empowerment issue-oriented
Collaborative
Change-oriented
Pragmatism
Consequences of actions
Problem-centered
Pluralistic
Real-world practice oriented
22
Several stances on philosophy in mixed
methods…
 One paradigm (pragmatism, transformative) (Tashakkori &




Teddlie, 2003; Mertens, 2003)
Multiple paradigms (dialectic perspective) (Greene, 2007)
Linking paradigms to design features) (Creswell & Plano
Clark, 2007)
Epistemological stance (ontology, epistemology, axiology,
methodology) (Guba & Lincoln, 2005)
Shared beliefs in a research field (Morgan, 2007)
 What it is
 How it informs your study
23
Make explicit your interpretive lens (theory)
 Advocacy lens (feminist, racial, ethnic, disability,
sexual orientation) (Mertens, 2003)
 Social science lens (social science theory)
 Components:
 What is it
 Who has used it in your field
 How it will shape your study (rephrase your guiding
research question, if a lens applies)
24
State your procedures (methods)
 Procedures for handling your qualitative and quantitative
data
 Sequence – concurrent or sequential or both
 Emphasis – emphasis on qualitative or quantitative
 Sometimes both concurrent and sequential phases are used
 Designs may include more than two phases
 Think about using a simple, elegant design
25
Parsimonious designs (Creswell & Plano Clark, 2007)
Concurrent Mixed Methods Designs
Triangulation Design
QUAN
Data &
Results
Interpretation
QUAL
Data &
Results
Embedded Design
QUAN
Pre-test
Data &
Results
Intervention
qual
Process
QUAN
Post-test
Data &
Results
Interpretation
26
Sequential designs
Explanatory Design
QUAN
Data &
Results
Following up
qual
Data &
Results
Interpretation
Exploratory Design
QUAL
Data &
Results
Building to
quan
Data &
Results
Interpretation
Sequential Embedded Design
Beforeintervention
qual
QUAN
Intervention
Trial
Afterintervention
qual
Interpretation
27
Design options
Design Name
Equal priority
QUAN emphasis
QUAL emphasis
QUAL+QUAN
QUAN+qual
QUAL+quan
n/a
QUAN(qual)
QUAL(quan)
Explanatory, sequential, quan first
QUANQUAL
QUANqual
quanQUAL
Exploratory sequential, qual first
QUALQUAN
qualQUAN
QUALquan
Concurrent, triangulation
Concurrent, embedded
Sequential, embedded
n/a
(qual)
QUAN
(quan)
QUAL
QUAN
(qual)
QUAL
(quan)
28
Explanatory sequential design
Quantitative
Data Collection
(quan)
Quantitative
Data Analysis
Case Selection
Qualitative
Data Analysis
(QUAL)
Quantitative Analysis
Case Selection
Qualitative Analysis
Interpretation –
based on quan
ad QUAL results
+
Qualitative
Data Collection
Quantitative Data*
Number of cigarettes
CES-D6
Qualitative Data*
Semi-structured
interviews, audio
recorded and
transcribed
* Data collected 10 times
over the course of a
calendar year for 40
participants
Graphic plot of CES
D6 scores over time
for each participant
Graphic plot of
cigarettes/day values
over time for each
participant
Selected 5 cases
maximally varying
Identified critical
months in which
smoking varied
Description of each
case
Identification of life
events occurring
during critical
months where
smoking increased or
decreased
Thematic analysis of
life events for each
case
Cross-case thematic
analysis
Source: Creswell, Plano Clark, Shope, McVea. (in progress)
Interpretation
Why did changes in
smoking occur?
Phase I Qualitative Research - Year 1
Qualitative Data Collection
Qualitative Data Analysis
Qualitative Findings
Phase II Quantitative Research - Year 2
Quantitative Instrument Development
Unstructured Interviews 50 participants
8 observations at the site
16 documents
Text Analysis: Using QSR N6
Development of codes and themes
for each site
Create approximately a 80-item
instrument plus demographics
Administer survey to 500 individuals
Quantitative Test of the Instrument
Quantitative Results
Determine factor structure of items and
conduct reliability analysis for scales
Determine how groups differ
using ANOVA test
Exploratory sequential design
Embedded research design
Experiment
QUAN
Data collection
Pre-test
Intervention
QUAN
Data collection
Post-test
Process – collection
and analysis of qualitative
data
(before, during, after trial)
32
Mixing the quan and qual data
Type of Mixing
Type of Design
Why Mixing
Occurs
Where Mixing
Occurs in
Research Process
Connecting
Sequential
One phase builds
on the other
Between data
analysis (Phase 1)
and data collection
(Phase 2)
Merging
Concurrent
Bring results
together
After analysis of
both quan and
qual – typically in
discussion
Embedding
Sequential or
Concurrent
Either building or
bringing results
together
Either between
phases or in
discussion after
analysis
33
Methodological issues
 Concurrent Designs
 Use strategies to explore
contradictory findings
 Use parallel questions
 Select sub-sample of
quantitative for qualitative
 Be sensitive to bias from
one data collection to the
other
 Sequential designs
 In Explanatory Design,
select qual sub-sample
from quan sample
 In Explanatory Design,
consider alternatives for
followup qual sampling
 In Exploratory Design,
samples can differ
 In Exploratory Instrument
Design, consider qual data
analysis approaches for
developing instrument
34
Write a purpose statement for a
triangulation design
This mixed methods study will address _________________ (overall
content-aim of the study). A triangulation mixed methods design will
be used, and it is a type of design in which different but complementary
data will be collected on the same topic. In this study, _________________
(quantitative instruments) will be used to test the theory of
_____________ (the theory) that predicts that __________________
(independent variables) will influence ________________ (positively,
negatively) the __________ (dependent variables or outcomes) for
____________ (participants) at __________ (the research site).
Concurrent with this data collection, qualitative _______________ (type
of qualitative data, such as interviews) will explore ____________ (the
central phenomenon) for _________________ (participants) at
_____________(site). The reason for collecting both quantitative and
qualitative data are to bring together the strengths of both forms of
research to ___________________ (e.g., compare results, validate results,
corroborate results).
35
Write a purpose statement for
a embedded design
This mixed methods study will address _________________ (overall
content aim of the study). An embedded mixed method design will be
used, and it is a design in which one data set provides a supportive,
secondary role in a study based primarily on the other data set. The
primary purpose of this study will use ________________ (quantitative
instruments) to test the theory of _____________ (the theory) that
predicts that __________________ (independent variables) will
influence ________________ (positively, negatively) the __________
(dependent variables or outcomes) for ____________ (participants) at
__________ (the research site). A secondary purpose will be to gather
qualitative data _______________ (type of qualitative data, such as
interviews) that will explore ____________ (the central phenomenon)
for _________________ (participants) at _____________(site). The reason
for collecting the secondary database is ________________ (e.g., to
address different question, to provide support for the primary
purpose).
36
Write a purpose statement for
an explanatory design
This study will address _______________ (content-aim of the study). An
explanatory mixed methods design will be used, and it will involve
collecting qualitative data after a quantitative phase in order to explain
or follow up on the quantitative data in more depth. In the first
quantitative phase of the study, ______________ instrument data to be
collected from _______________ (participants) at ___________ (research
site) to test _______ (the theory) that explains why ______________
independent variables) relate to the ____________ (dependent
variables). The second qualitative phase will be conducted because
______________ (intent of the qualitative phase). In this exploratory
follow-up, the ______________ (central phenomenon) will be tentatively
explored with ___________ (participants) at _____________ (the research
site). The reason for the exploratory follow-up is to _______________
(e.g., to help explain or build upon initial quantitative results).
37
Write a purpose statement for
an exploratory design
This study addresses _________________ (content-area of the study). The
purpose of this exploratory sequential design will be to __________________
(e.g., develop an test an instrument, generate a taxonomy). The first phase
of the study will be a qualitative exploration of ________________ (the
central phenomenon) by collecting ___________________ (types of data)
from _________________ (participants) at _____________ (the research site).
The second quantitative phase will follow up on the qualitative phase for
the purpose of __________(intent of this followup). In the quantitative
phase, ___________ (instrument data) will be collected from ____________
(participants) at ______________ (research site). Quantitative research
questions/hypotheses will be formulated after the completion of the initial
qualitative phase. The reason for collecting qualitative data initially is that
_____________________ (e.g., instruments are not available, variables are not
known, there is little guiding theory).
38
Qualitative research questions
 Qualitative central question
 Begin with “what” or “how”
 Focus on single phenomenon
 Use exploratory verbs (discover, understand, explore)
 Non-directional language
 A general question (allowing participants’ perspectives
to emerge)
39
Quantitative research questions
 Can be hypotheses or questions
 State variables – independent, dependent, mediating,
covariates
 Develop from theory
 Use distinct measures for independent and dependent
variables
 Order variables from independent to dependent
40
Writing research questions/hypotheses in
mixed methods research
 Write qualitative research questions and write quantitative
research questions/hypotheses
 Also write a mixed methods research question
 Write these questions separately
41
A new type of research question: A mixed
methods question
Three ways to write this question:
Methodologically-focused:
 To what extent do the qualitative results confirm the quantitative
results?
Content-focused:
 How do the interviews with adolescent boys support the
quantitative results that their self-esteem changes during the
middle school years?
Hybrid of quantitative and qualitative elements:
 What results emerge from comparing the exploratory qualitative
data about boy’s self-esteem with outcome quantitative
instrument data measured on a self-esteem instrument?
Order the topics for your plan








Title
Introduction
 (Problem)
 Worldview/theoretical lens
 Audience
 Purpose Statement
 Research Questions
(Literature Review)
Methods
 Type of Mixed Methods Design (also add definition of mixed methods)
 Types of Data Collection
 Types of Data Analysis

Sequence/Emphasis/ Mixing Procedures

Figure of procedures
 Anticipated methodological issues
Ethical issues anticipated
Validity issues
Researcher resources and skills
References, Appendices
43
Additional resources
Books:







Creswell, J. W., & Plano Clark, V. L. (2007). Designing and
conducting mixed methods research. Thousand Oaks, CA:
Sage.
Creswell, J. W. (2008). Research design: Qualitative,
quantitative, and mixed methods approaches (3rd ed.).
Thousand Oaks, CA: Sage Publications.
Greene, J. C. (2007). Mixed methods in social inquiry. San
Francisco: Jossey-Bass.
Mertens, D. M. (2005). Research methods in education and
psychology: Integrating diversity with quantitative and
qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage
Publications.
Plano Clark, V. L., & Creswell, J. W. (2008). The mixed
methods reader. Thousand Oaks, CA: Sage.
Tashakkori, A. & Teddlie, C. (1998). Mixed methodology:
Combining qualitative and quantitative approaches.
Thousand Oaks, CA: Sage Publications.
Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of
mixed methods in social and behavioral research. Thousand
Oaks, CA: Sage Publications.
44
Additional resources
Articles and Chapters:







Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixedmethod evaluation designs. Educational Evaluation and Policy Analysis, 15 (2),
195-207.
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003).
Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie
(Eds.), Handbook of mixed methods in social and behavioral research (pp. 209240). Thousand Oaks, CA: Sage.
Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological
issues in conducting mixed methods research. In M.M. Bergman (Ed.),
Advances in mixed methods research. London: Sage.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual
framework for mixed-method evaluation designs. Educational Evaluation and
Policy Analysis, 11 (3), 255-274.
Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods
sequential explanatory design: From theory to practice. Field Methods, 18(1),
3-20.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of
Mixed Methods Research, 1, 48-76.
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological
triangulation. Nursing Research, 40, 120-123.
45
by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln, and Founding Co-Editor, Journal of
Mixed Methods Research
University of Pretoria, October 21, 2008
© Please do not duplicate or use these slides without the express permission of the author.
46