Transcript OPEN AIR Research

OPEN A.I.R
Strengthening
Research
Methodologies
Donna Podems, PhD, MPA
Christa Oosthuizen, MA
Introduction
 Objective:
To have improved written
research methodologies for your case
studies.
 This
will lead to a stronger presentation of
your methodology sections that will result
in more credible research findings
 It
may result in additional field work
Working Style for the Workshop
 Discuss
one topic at a time
 Provide
time to fix your section, make
notes, ask questions, discuss
 If
your section is ok (e.g. sampling is clear)
then please use this time to catch up on
emails, work, calls, etc. SERIOUSLY!
Case Study Research
 Case
study is not a methodological
choice but a choice of what is to be
studied
 Well-constructed case studies are holistic
and context sensitive
Case Study Research
 According
to Robert Stake (1995)
“Case study is the study of the particularity
and complexity of a single case, coming to
understand its activity within important
circumstances.”

The case is a specific, complex, functioning
thing.
Case Study Research
 We
are interested in cases for their
uniqueness and commonality
 We
seek to understand them
 We
would like to hear the stories
 We
are interested in how something
functions, and we need to put aside
many presumptions while we learn (Stake, 1995)
Case Study Research
 Intrinsic
Case Study - we are interested
in it not because studying we learn
about other cases, but because we
need to learn about THAT particular
case
 Instrumental case study—when we
have the need for a general
understanding
 Collective case study -
Research Reports
 Introduction
–set the scene and research
questions
 Literature Review – what is out there
already and refine research questions
 Description of Methodology
 Discussion of Results
 Conclusion-recommendation-way
forward
Methodology
Just Some Basics

Methodology comes at the start before you
describe a method

Methodology vs. method
Questions and Boundaries
•
Research Questions
 Most of these are already set and clear
 Do not confuse the research question with the
questions that you ask the interviewee
 All decisions link back to your questions
 Confusion between action and question
•
Boundaries
Do you clearly state your research boundaries?
Unit of Analysis
 No
matter what you are studying, always
collect data at the lowest level unit of
analysis possible
 Collect data about individuals not
households
 You can always aggregate, but you can
never disaggregate data collected in
groups
Unit of Analysis
 What
is the phenomena you are
studying(Subject of your case)?
Caution: Must be real life and not
something abstract, argument or hypothesis
 What
case)
is the context (Data external to your
Unit of Analysis
Literature Review
Literature Review – What Is It?
“A literature review is an effective
evaluation of selected documents on a
research topic that should provide
background to the proposed study.” Hart,
1998
Literature review - Steps to
Follow and Describe
1.
2.
3.
4.
5.
6.
7.
Defining parameters
Search for literature
Sort and prioritise retrieved literature
Analytical and evaluative reading of
literature
Comparison across studies
Organising the content
Writing the review
Literature reviews –Define
Parameters?
 Language
 Subject
area
 Geographic area
 Sector
 Publication period
 Literature type
Literature reviews – What to
Aim For
 Summarize,
 LOOK
Analyze, Interpret ---refer to!
AT: Research reports and journal
articles in the field to see how other
researchers have described and
evaluated the literature, and to examine
how they have ordered and classified
their findings and the findings of others.
The Methodology Section:
Why Have One?
 From
the description of the methodology,
the reader should know exactly what was
done so that
 It can replicated and get similar results
(internal validity)
We need to ensure that your methodologies
meet this requirement
Revisiting Your Methodology

These are already written –we need to
improve them.

What belongs in a methodology section?







Methods of data collection
Sampling
Research instrument and how it was designed
Data analysis
Process – time frame
Ethical considerations
Weaknesses to design
What Does NOT Belong in a
Methodology Section?
We found some things that didn’t belong.
These were:
 Data
 Results
 Findings
Please review your case studies and
highlight or remove these items
Methods
 Interviews
 Focus
groups
 Direct observation
 Surveys
 Documentary review – literature, policy
documents (secondary data)
Interviews
 Structured,
semi-structured, open ended
questionnaires:

Depending on what you use, your reason
for that, and your data analysis would
change
Please review your case studies and describe
these tools and who they were used for
(which group)
Focus Group
Focus Group
 Why
use a Focus Group and not an
interview?




Please don’t say time was the factor
Want several perspectives at the same time
Discussion and group interaction will
produce the most useful data
It is a way of listening and learning from
discussion
Direct Observation
 Not
as easy as it seems….
 Not appropriate for these studies
Brief demonstration
Questionnaires for Interviews
 Structured
 Semi
structured
 Unstructured
Questionnaires for Surveys
 Surveys
are used to collect primary
quantitative and qualitative data with the
aim of gathering valid, reliable, unbiased
and discriminatory data from a
representative sample of respondents to
obtain generalized results that is
applicable to the population.
Document Review

Different than a literature review

Secondary data- may still need to analyse

Assessment of the quality, novelty, and
importance of the article

Need to clarify why you chose certain data
and why you didn’t review other data
Sampling- Selecting a Case
 Might
be useful to select cases which are
typical of other cases but a sample of a
few is unlikely to be a strong
representation of others
 Case study research is not sampling
research
 We do not study a case primarily to
understand other cases
Sampling- Selecting a Case
 Our
first obligation is to understand this
ONE case
 Sometimes
a typical case works well
 Sometimes
an unusual case illustrates
matters we would otherwise over lookHow did you select your case?
Sampling- Selecting a Case
 Maximise
 Feasible
 Not
what we can learn
and accessible
all cases will work out well
Sample
Qualitative –Thoughts
Sampling
Qualitative – Who To Interview
 Extreme
Case Sampling – weakness is lack
of generalising
 Intensity
Sampling – Same as extreme with
less emphasis on the extreme
 Maximum
variation sampling – any
patterns that emerge from great variation
are of interest
Sample
Qualitative – Who To Interview
 Homogeneous
samples – direct contrast
to max variation. Purpose is to describe
some small group in depth
 Typical
case sampling – describe a
culture or program; a qualitative profile
 Critical
case sampling- those that make
the point quite dramatically
Sample
Quantitative – Who To
Interview
 Snowball
or chain sampling- information
should diverge and then converge
 Criterion
sampling- review and study all
cases that meet some predetermined
criterion of importance --Critical incidence
 Theory-based
sampling – sample based on
potential manifestation or representation of
important theoretical constructs
Sample
Qualitative – Who to Interview
 Confirming


and disconfirming cases-
Confirming is to identify additional
examples of already identified patters;
elaborate and , add richness, depth and
credibility
Disconfirming are examples that do not fit;
place boundaries around the findings –
Who has this in their Case Study?
Sample
Qualitative– Who to Interview
 Stratified
purposeful sampling- sample
within a sample. Stratify within a larger
case
 Opportunistic or emergent sampling- new
opportunities after field work has begun
 Purposeful random sampling- random
sample of even a small case can increase
credibility—not for representativeness
Sample
Qualitative – Who to Interview
 Sampling
politically important cases
 Convenience sampling – what’s fast and
convenient
 Key
to deciding which one? Select
information rich cases; cases from you
can learn a great deal about matters of
importance and worthy of in-depth study
Sample
Qualitative – Sample Size
 What
is the appropriate number to
sample? It depends
 There
are no rules for sample size in
qualitative inquiry.
 Credibility
is –does the sample strategy
support the study’s purpose? For some,
redundancy in the data, for others…
Sampling Genie
Before Moving On To
Quantitative Sampling….
 Choice
between quant and qual well
explained in your methodology?

Quant want explanation and control
(explain) while Qual want understanding for
the complex interrelationships among all
that exists (inquiry)
Check to see that you have this well
explained
Sampling
Quantitative

Probability - random selection




Large numbers
Known population
Large scale surveys using quant technique
(usually)
Non probability – selection is not chance
All have weaknesses (to address later)
Data Analysis
Qualitative
 Need
data



to explain how you analysed your
Organise by cases
Can shift your unit of analysis at this stage
Rubrics, computer programs, hand sorting
Who needs to go back and change their unit
of analysis after they analyse the data?
Data Analysis
Qualitative

Content, Pattern and Theme core
consistencies
Inductive to find theme and then deductive
to confirm the analysis ---needs to be
explained in your methodology.
Data Analysis
Quantitative

In quantitative research, you end up with "numbers"
after carrying out your research. These are analyzed,
and then interpreted in light of the research question
and other relevant theory and research findings.

In order to create the "numbers" for quantitative
research (data), a measurement process needs to
take place. In other words, you need to convert some
human phenomenon (in the human sciences)
accurately into numerical data.


Descriptive
Multivariate
Credibility
 Weakest
part of most methodologies
 Rationale
---why is this research credible?
 Credible,
transferable, dependable,
confirmable
 Transparent
methodology
Comparative Study – What
Are We Comparing?
Your bright red skin is a delight to see. But
the seeds inside are what make you like me.
As a child I was told never to dare, an
apple and an orange one could not
compare. To find we are alike I scarcely
can bear. Don’t tell me we also are like a
green pear
Comparative Study
 Comparative
approach?


studies: why are using this
Comparative study can use either
qualitative or quantitative methods
Whatever the methods used, comparative
research that crosses national boundaries
increasingly takes account of socio-cultural
settings
 Examples
be…
of what you can write would
Comparative Study

…Comparisons have served as a tool for
developing classifications of social phenomena
and for establishing whether shared
phenomena can be explained by the same
causes.

…Provide an analytical framework for
examining (and explaining) social and cultural
differences and specificity.

…Contextualisation, cross-national comparisons
have served increasingly as a means of gaining
a better understanding of different societies,
their structures and institutions.
Comparative Study
If you are using Comparative Approach…
please take a moment and think about how
to justify WHY you are choosing that
approach.
Are you also looking at causes,
consequences and relationships?
Data Triangulation
 Data

triangulation
Variety of data sources
 Investigator

triangulation
Several researchers
 Theory
triangulation
 Use
of multiple perspectives to interpret a single
set of data
 Methodological

triangulation
Multiple methods
Triangulation
 Brings
credibility
Focus
Groups
Data
Triangulation
Documents
Interviews
Triangulation
 Brings
credibility
Analysis 3
Data
Triangulation
Analysis 1
Analysis 2
Data Analysis
Mixed Methods
 Demonstrate
together
 Is
how they are being used
one primary?
 What
do you do when one provides the
opposite information?
Process
 When
whom
 Most
 It
the study was conducted and by
methodologies are missing a ‘flow’
should read as a part of your story
Ethics
 Used
consent form
 Approval for study (IRB)
 Ethics considered
Are these clearly articulated in your
methodology?
Weaknesses of your study
 What
are the weaknesses? This is not a
‘weakness’ to do write this. All studies
have weaknesses.
 Please



review your methodology.
What are the weaknesses?
Are they clearly explained?
Are they reasonable?
Having Said All That
 Only
small portions or a truncated
methodology may be in the final book
 However
being able properly defend the
research is critical, especially if this
research continues
Final Words…
Final Thoughts