Transcript Slide 1
An Introduction to
Qualitative Research
Barbara Pini
John Curtin Institute for Public Policy
Curtin University of Technology
[email protected]
Overview
Part One: Research Process and Design
Part Two: Sampling
Part Three: Documentary Research
Part Four: Participant Observation
Part Five: Interviews
Part Six: Focus Groups
Part Seven: Mixed Methods (Case Studies, Ethnography and Action
Research)
Part Eight: Qualitative Data Analysis
Part Nine: Trustworthiness in Qualitative Research
Part Ten: Ethics and Qualitative Research
PART ONE
Research Process and Design
What is Research?
“A systematic process of critical enquiry
leading to valid propositions and
conclusions that are communicated to
interested others” (McLeod, 1994).
What are some of the key words in this definition and
why are they important?
Research Proposal
Why do we need it?
At Curtin University you will need to produce a
written document outlining your research for the
Faculty of Humanities. This is called an
Application for Candidacy.
Such proposals are very common in academia
(for funding, ethics approval etc).
Components of Research Proposal
BACKGROUND
AIMS/OBJECTIVES/RESEARCH QUESTIONS
SIGNIFICANCE
METHODS
ETHICS
FACILITIES AND RESOURCES
TIMELINE
Taking A Closer Look at Methods
METHODS
What methods will you use to address the research questions?
How many and why this many? (sampling)
How will these methods be designed? i.e. How will the study be
conducted? Where? How will you gain access?
What is the justification for these methods?
What questions will be asked and why?
What are the limitations of these methods and how will you address
these limitations?
How will analysis be undertaken?
What are the ethical concerns related to these methods and how will
these be addressed?
All the methodological decisions you make – i.e. how you answer each
of the above questions should be tied to the methodological literature
and/or the literature in your subject area.
Deciding on a methodological
approach
Ontology: What is the nature of the phenomena,
or social reality, that you want to investigate?
Epistemology: What might represent knowledge
or evidence of the social reality that you want to
investigate?
Research area: What topic is the research
concerned with?
Research Question: What do you wish to
explain or explore?
Ontology
What is the nature of things in the social world?
For example, are you investigating:
Bodies, subjects, objects
Rationality, emotion, thought
Feeling, memory, senses
Motivations, ideas, perceptions
Attitudes, beliefs, views
Texts, discourses
Cultures, society, groups
Interactions, social relations
Some ontologies are better matched to qualitative research
methodology than others (e.g., social processes, interpretations,
social relations, experiences etc.)
Epistemology
What is your theory of knowledge? What are your presuppositions about
the nature of knowledge?
Examples of epistemological perspectives
Positivist Perspectives (also called empiricism)
Fundamental claim is that reality is a fixed, measurable entity that
is external to people.
There exist “social facts.”
Aims to to find true, precise and wide-ranging laws of human
behaviour which we can generalise to the population as a whole
“If it can’t be measured, it doesn’t exist.”
Social Constructionism
Reality is constructed socially so rejection of “social facts”
Aim is to describe the subjective and consensual meanings that
constitute social reality.
Understanding of social world as “local truths” which cannot be
evaluated by external criteria
Qualitative and Quantitative
Methods
Qualitative
Makes less use of mathematical
techniques.
Focus on interpretation by
researcher
Systematically arranging and
presenting information to search
for meaning in data collected
“Words, not numbers”
Usually involves a philosophical
stance that human knowledge is,
to some extent, contextualised or
local.
But some form of counting is
almost always involved in
qualitative analysis.
Quantitative
Employs statistics or other
mathematical operations to
analyse data
Concepts are assigned numerical
values
Collects a small amount of data
from a large number of people
Allows generalisation to wider
population
Strengths of Quantitative Research
It can deal with large numbers of cases
It is capable of examining complex
patterns of interactions between variables
It can make possible the verification of the
presence of cause and effect relationships
between variables
Weaknesses of Qualitative
Research
Lack of in-depth information
Ignores individual perspectives and experiences
Limited with topics we know little about
Can be built on pre-existing biases of the researcher
The case of questionnaires:
Language used
Ordering of questions
Forced response formats; what if ‘it depends…’?
Missing data
Sampling issues
Response rates
Lies, lies and damn statistics; torturing your data until it confesses
Strengths of Qualitative Research
Research done in natural settings
Emphasis on informant interpretations and
meanings
Seek deep understanding of informants world
“Thick Description” (Clifford Geertz)
Humanising research process by raising the role
of the researched
High levels of flexibility in research process
Weaknesses of Qualitative
Research
Problems of reliability - The difficulty of replicating findings
“Subjectivity” of nature of data collection and analysis
Observations may be selectively reported making it impossible to
gauge the extent to which they are typical
Risk of collecting meaningless and useless information from
participants.
Problems of objectivity vs detachment (particularly in participant
observation but also applies to other methods)
Problems of ethics: Entering the personal world of the participant
Very time consuming
PART TWO
Sampling
Why Sample?
1. Generalisability
To
generalise the properties of the sample to the wider
population
To make conclusions regarding the wider population
2. Pragmatic reasons
applied research in organisations often has resource
constraints
sampling reduces burden on resources (i.e. time and
money)
usually not feasible to contact the whole population
3. Destruction of test units
some research projects (e.g. quality testing) require
the destruction of the items being tested e.g. testing
cars for safety
Sampling
Important issues:
properly
selected samples are sufficiently
accurate in most cases to make statements about
the population
even when population has considerable
heterogeneity, larger samples can provide data of
sufficient precision upon which to base
conclusions
In quant research, the characteristics of the
sample and actual sample size is more important
than the relative size of the sample compared to
the population
In qualitative research, aim is not statistical
representativeness, but representativeness in the
sense of gaining access to the full range of views,
themes or possibilities in the population
Ensuring Representativeness
Once the decision to sample has been made, the
researcher must identify the target population
Must carefully define the target population so that the
proper source from which to collect the data can be
identified
Population
any complete group sharing some common set of
characteristics
can be finite or infinite
the group we wish our research to comment upon
Types of Sampling
Probability Sampling
Probability sampling
facilitates
generalisability
occurs when elements in the population have an
equal probability of being selected in the sample
logic depends on selecting a truly random and
statistically representative sample that will permit
confident generalisation from the sample to a larger
population
best for quantitative research
Types of sampling
Probability sampling
How do you select a sample that will look like
the population in terms of its demographics?
Simple
random sampling
Systematic sampling
Stratified random sampling
Cluster sampling
Area sampling
Double sampling
Advantages and Disadvantages of
Sampling Methods
Simple random sampling
high generalisability of findings
not as efficient as stratified sampling
Systematic sampling
easy to use
systematic biases are possible
Stratified random sampling
most efficient and precise
would adequately represent strata with low numbers
Advantages and Disadvantages of
Sampling Methods
Cluster sampling
goal is to reduce costs of data collection
the least reliable among all the probability designs
Area sampling
type of cluster sampling
cost-effective. Useful for decisions regarding location
Double sampling
offers more detailed information on the topic of study
original biases, if any, will be carried over
Types of sampling
Non-probability sampling
Non-probability sampling
occurs when elements in the population do not have a
pre-determined probability of being selected in the
sample ie non-random
logic depends on selecting cases rich in information
that will permit an in-depth understanding of the
research question
often used in qualitative research
an un-representative sample might be a useful and
more stringent test for a law-like hypothesis
E.g. Does wealth = health?; sample the very rich who should
be extremely healthy and the very poor who would be
extremely unhealthy
E.g. Helping behaviour and number of onlookers?; sample
disaster situations and assess whether ‘law’ holds
Convenience Sampling
Convenience sampling
e.g.,
interviews on the street; simply asking for
volunteers; using clients in clinical or business
setting
quick, convenient, less expensive
not generalisable at all
Purposeful Sampling
Purposeful sampling or Judgment sampling
sometimes
the only meaningful way to investigate
Useful when you need a targeted sample
Includes:
Snowball sampling
Starting with a small group and asking for further contacts
Useful for sensitive topics
Quota sampling
Population is stratified and numbers within strata are decided
Contacts are made until quotas are full
Quotas can be proportional or non-proportional to the
population
Sampling Criteria
The appropriate sample design will depend on the
following criteria:
degree
of accuracy required
local
versus national project
need
for statistical analysis
resources
(ie time / money)
Sample Size in Qualitative Studies
Adequacy of sample depends not so much
on the number of cases
Depends on the proper specification of the
cases to be analysed
Redundancy in information is often a sign
that the sample size is adequate
PART THREE
Documentary Research
DOCUMENTARY RESEARCH
“The good stuff of social science”
(Ryan & Bernard 2003)
Definition
Types of documents
Classifications
Advantages and disadvantages
Analysis
Ethics
Activities: Journal Article and Document
analysis
Definition
A document in its most general sense is a written
text…Writing is the making of symbols representing
words, and involves the use of a pen, pencil, printing
machine or other tool for inscribing the message on
paper, parchment or some other material
medium…Similarly, the invention of magnetic and
electronic means of storying and displaying text should
encourage us to regard ‘files’ and ‘documents’ contained
in computers and word processors as true documents.
From this point of view, therefore, documents may be
regarded as physically embodied texts, where the
containment of the text is the primary purpose of the
physical medium
(J. Scott 1990: 12-13)
So…this includes…
Photographs
Videos / film footage
Political speeches
Minutes of meetings
Plays, novels
Media sources
Personal documents such as diaries, oral histories
Emails
[Visual]
Any other suggestions?
Classification of Documents
Primary
Written
or collected by those who actually witnessed
the events they describe.
Secondary
These
are written after an event which the author had
not personally witnessed.
Tertiary
These
enable us to locate other sources. They are
indexes, abstracts, bibliographies etc.
Other Typologies for classifying Documents
John Scott (1990) divides documents into four
categories according to the degree of their
accessibility:
Closed
Restricted
Open-archival
Open-published
Public and Private documents
Solicited and unsolicited documents (Some
documents are produced with the aim of
research in mind, whereas others would have
been produced for alternative uses).
Advantages of Documentary Analysis
Cost-effective
Permanence – particularly for past
events
Access is usually relatively easy
Provides understanding of certain
phenomena that is rich in detail and
meaning
Methodological Problems
Authenticity of documents
Secondary data
When researchers use documents as a source of data, they generally
rely on something which has been produced for other purposes and not
for the specific aims of the investigation.
Credibility
What purpose was the document written to? Who produced the
document? What was the status of the author? When was the document
produced?
Representativeness and bias
Is the document typical of its type? Does it represent a typical instance
of the thing it portrays? Is it complete? Has it been edited?
Understanding meaning
Is the meaning of the words clear and unambiguous? Are their hidden
meanings?
Reading/legibility
Incomplete sources
Gaining access (restricted documents)
Analysing Documents
Quantitative approaches
Content analysis
Qualitative approaches
“Reading” the text
An understanding of the context in which it was
produced
Examining symbols, hidden meanings
What is not contained in the text? What does this
mean?
Depictions of work in family-genre movies
PART FOUR
Participant Observation
Definition
By participant observation we mean the
method in which the observer participates in the
daily life of the people under study, either openly
in the role of researcher or covertly in some
disguised role, observing things that happen,
listening to what is said, and questioning people,
over some length of time.
(Becker and Geer 1957: 28)
Origins and Links
Origins in ethnography
Linked with epistomological orientations of
ethnomethodology and grounded theory (these methods
entail naturalistic investigations of culturally contexted
social processes).
Pseudo-objective stance of the researcher has largely
been abandoned in favour of more personal and
subjective accounts of the participant observation
experience (see Tedlock, 2000)
Traditionally this method has been paired with interviews
and document analysis, and more recently with digital
photography
When to use participant observation
Participant observation is especially appropriate for
scholarly problems when:
Little is known about the phenomenon (a newly formed
movement/religion)
There are important differences between the views of insiders as
opposed to outsiders (e.g. labour unions and management)
The phenomenon is somehow obscured from the view of
outsiders (mental illness, family life, private interactions)
The phenomenon is purposefully hidden from public view (crime
and deviance, secretive groups)
Strengths of participant observation
Natural/unobtrusive.
Requires little more than self
Can produce rich insights into complex realities
Context specific and flexible
Holistic. Can incorporate relationships between factors
(people, settings, documents).
Provides insight into actors’ meanings as they see them
Offers advantage of serendipity
(See Dennis 1993).
Limitations of participant
observation
Access. Limited options open to the researcher
about which roles to adopt or settings in which to
participate
Commitment. Demanding method and
significant personal resources.
Danger (potentially)
Reliability
Observer effects
Representativeness of data. Difficulty of
generalising from data
Ethical issues
Easy or Difficult?
This method (participant observation) is one that
those new to social research believe they can
undertake with ease. On first glance it appears
to be just about looking, listening, generally
experiencing and writing it all down. However, it
is more plausible to argue that participant
observation is the most personally demanding
and analytically difficult method of social
research to undertake
(May 2001: 153).
Strategies to overcome limitations
Use multiple observers or teams
Search for negative cases
Spend an extended time in the field
Use insider checking
Use outsider checking
Repeat observations under varying conditions
Be meticulous in recording observations
(Alder and Alder 1994)
Participant Roles
Complete
participant
Enter the field under pretence or deception
Engages fully in the activities of the group or
organisation under investigation
Advantages are that it can produce more
accurate/authentic information and an
understanding not otherwise available
Problem of recording observations
[Visual]
Gold (1958)
Types of Participant Observation
Participant as observer
Enter the field setting with
an openly acknowledged
investigative purpose.
Develop relationships with
subjects
Problem of ‘going native’
but dismissed by some
(e.g. May 2001)
May encounter hostility –
particularly in early stages
of research
Problem of
disengagement from field
[Visual]
Observer as participant
Strictly speaking this
would not be regarded as
participant observation
No lasting contact with
people
Focus on observation, not
on interaction with people
Problem is that it does not
utilise the strengths of the
time in the field to deepen
understanding
Participant Roles
Complete
observer
Also a non-participant role
Role completely removes the researcher from
observed interactions
Epitomised by laboratory experiments
[Visual]
Stages in Participant Observation
Denzin (1989)
Before actual field contacts and observations begin, a
general definition of research problem is identified.
Select field setting.
Make initial contact and establish access.
Collect descriptive data on setting and participants.
Field work progressing. Informants selected, approached
etc. Early theoretical formulations tested.
General categories for data analysis are developed.
Refining observations.
Complex set of propositions developed and tested.
Conclusion of study. Role disengagement. Writing of
report.
Recording Observations
Spradley (1980) and Jorgensen (1989)
discuss initial observations as primarily
descriptive, unfocused and general.
After observers become more familiar with
their setting and grasp the key aspects of
this setting their observations will become
more focused and selected.
Recoding Observations
The participants: Who are the participants? How many are there? How can
they be characterised (gender, occupations etc) Where are they situated in
relationship to each other? Are there any key groupings or relationships?
The tasks: What are the functions of the various groups of people? How are
they relating in this setting? What are they doing during the key events or
observations? Are these functions formally defined? Do individuals and
groups have a variety of purposes for being there? Are there conflicting
goals of various groups or individuals? What are these conflicting goals?
The setting: Each setting has unique features. What are these? Equipment?
Resources? Facilities? Use your senses.
The behaviour and the outputs: How do people actually behave during the
event? Describe this behaviour in descriptive terms. What are the specific
movements made and activities that are carried out?
Timing: The timing of the behaviour is described by the time it occurred, the
time it takes, and the frequency.
Unique causes or consequences: What unique occurrences affected the
people, tasks, setting, behaviours, output and timing?
Cunningham (1993:141)
Video recording
A “privileged gaze” (Atkinson & Hammersley,
1994).
Purposes (Paterson, Bottorff & Hewat, 2005):
Allows decontextualised sequencing of minute behaviours,
concurrent behaviours, nonverbal behaviours and conversational
analysis that are difficult to observe in real time
To document the research process and check for observer effects
To direct methodological decisions
To enhance the validity of the researcher’s interpretation of
observations
Compared to participant observation where videorecordings are not used, relationships less
important for the collection of data, but more
important for getting consent to participate
Participant Observation and Ethics
Disguised or covert research has come
under significant attack.
Boundaries between covert and open
research are not necessarily clear cut.
Protection of informants/respondents
Why have people shared information?
Ethical dilemmas do not cease when you
leave the field.
PART FIVE
Interviews
Interviews 1
Defining Interviews
Types of Interviews
Advantages and
disadvantages
Design questions
Sampling issues
Types of interview questions
Interview skills
Defining Interviews
A conversation with a purpose (Kahn and
Cannell 1957:149)
Silverman (1993) talks about us living in
an ‘interview society’
Estimated that 90 per cent of all social
science investigations use interviews in
one way or another (Briggs 1986)
Types of Interviews - Structured
Many are formally structured.
Associated with questionnaire research (oral questionnaire); also
used in some job interviews
Each person asked the same question in the same way so that
any differences between answers are held to be real ones and
not the result of the interview situation itself.
No deviation from question order or wording of questions.
No adjusting for level of language.
No clarifications or answering of questions about the interview
Types of Interviews: Semi-structured
Questions
are normally specified, but the interviewer
is freer to probe beyond the answers.
Questions may be reordered during the interview.
Level of language may be adjusted.
Interviewer may add or delete probes.
Allows people to answer more on their own terms, but
still provides a structure for comparability.
Sometimes called semi-standardised.
Most typically used in qualitative studies (Rossman
and Rallis 1998: 124)
Types of Interviews: Unstructured
Includes
life-history, biographical and oral
history interviews
Sometimes called informal, non-standardised
Provides qualitative depth in allowing subject
to talk about topic within their own frame of
reference
Issues
“Increasingly, qualitative researchers are
realizing that interviews are not neutral
tools of data gathering but active
interactions between two (or more) people
leading to negotiated contextually based
results” (Fontana and Frey 2003: 62).
Impact of identities of researcher and
participants should be considered
Advantages of interviews
One of the most flexible/responsive methods available as
different types of interviews can be engaged for different
research problems.
Ability to explore additional research questions / issues if they
arise (semi-structured / unstructured only)
Ability to gain rich and descriptive data; ideally suited to
examining topics in which different levels of meaning need to
be explored.
Most participants will accept an interview readily. They are
likely to be familiar with interviews.
Ability to follow up research participants for clarification or
further exploration
Disadvantages of interviews
Bias and subjectivity which, in turn, affects validity and reliability
of data
Generalisation problem
Process of data collection, transcribing and analysis from each
participant time-consuming; thus, sample size generally not
large
In reporting results, tendency of researchers to focus on quotes
which are dramatic, unusual or interesting, rather than typical
Design Questions
The recommended duration for an in-depth interview is
one hour and a half, but may be varied according to the
situation and respondent (Burgess 1984, p.120).
A write-up of observations may be completed following
each of the interviews (Burgess 1984, p.119).
Increased rapport is likely to be facilitated through
follow-up visits which also will improve the quality of data
produced (Whyte 1984, p.114; Lee 1993, p.113).
What Counts as Data?
Utterances only?
Non-verbal aspects of the interaction?
Written notes / tape-recordings?
My own memories and unwritten interpretations
of the interview?
Diagrams, pictures, drawings, charts and
photographs produced during the interview?
NB: Absolute objectivity is a myth!!!
Researchers continually make judgements
about what to write down or record, what they
have observed, heard and experienced and
what they think it means (Mason, 2002).
Sampling (as per Part 2)
Specifically, Minichiello et al. (1995, p.
162) describe the process as it applies to
in-depth interviewing as ‘selecting
informants on the basis of relevant issues,
categories and themes which emerge in
the course of conducting the studies’.
Types of Interview Questions
(Kvale 1996: 133)
Introducing questions
E.g. “Can you tell me about…”? Etc.
Probing questions
E.g. “That’s interesting. What else can you tell me about…”?
Specifying questions
E.g. “Can you give me an example of…”?
Direct questions
E.g. “Earlier you said… How does that relate to…”?
These may need to come later in the interview; may be slightly
confrontational or ask for clarification of discrepant information
Indirect questions (useful when trying to avoid social desirability bias)
E.g. “What should someone else in that situation do…”?
Structuring questions
E.g. “I would now like to introduce a new topic…”
Silence – just a nod or a pause
Interpreting questions
Rephrasing an answer, more speculative questions
E.g. “So does that mean…”?; “Are you saying…”?; “Would I be right in
interpreting that as…?”
Interview Skills
The good interviewer needs to be attentive.
The good interviewer is sensitive to the feelings
of the informant.
The good interviewer is able to tolerate silence.
The good interviewer is adept at using prompts.
The good interviewer is adept at using probes.
The good interviewer is adept at using checks.
The good interviewer is non-judgemental.
(Denscombe 1999:135)
PART SIX
Focus Groups
Overview
Definitions
History
Common uses
Advantages and limitations
Interviews versus focus groups
Recruiting for a focus group
The role of moderator
Analysing focus group data
Ethics of focus group research
The future?
Activity
Definitions
‘The hallmark of focus groups is the
explicit use of the group interaction to
produce data and insights that would be
less accessible without the interaction’
(Morgan 1988: 12).
Kitzinger (1994: 159) ‘group discussions
organised to explore a specific set of
issues’.
History
Originally called focussed interviews.
Origins in the Office of Radio Research at Columbia
University in 1941 when Paul Lazarsfeld invited Robert
Merton to assist him in the evaluation of audience
response to radio programs.
Method increasingly used in social sciences and
marketing (Catterall and Maclaran 1997; Green 1999).
Morgan (1988) says most common form of marketing
research.
Common uses of focus groups
Obtaining general background information about a topic
of interest
Generating research hypotheses that can be submitted
to further research and testing using more quantitative
approaches (Stimulating new ideas and creative
concepts)
Diagnosing the potential for problems with a new
program, service or product
Generating impressions of products, programs, services,
institutions or other objects of interest.
Learning how respondents talk about the phenomenon
of interest. This, in turn, may facilitate the design of
questionnaires, survey instruments or other research
tools that might be employed in a research project.
Interpreting previously obtained research results.
Advantages of Focus Groups (Quible 1998,
Albrecht, Johnson and Walther 1993; Stewart and
Shamdasani 1990).
time and cost efficient
direct interaction between researcher and researched, respondents can
qualify responses, researcher can observe non-verbals
large amounts of rich data in the respondents’ own words. The researcher
can obtain deeper levels of meaning etc.
synergistic
flexible
especially useful for groups with limited literacy,
results are readily understood.
synergism
snowballing
stimulation
security
Spontaneity
Disadvantages of focus groups
(Quible 1998, Albrecht, Johnson and Walther 1993; Stewart and
Shamdasani 1990).
Small number of respondents and convenience recruiting limit
generalisability.
Responses may be subject to group-think, especially if there are
dominated or opinionated members. More reserved members may
be overlooked (see MacDougall and Baum 1997).
The open-ended nature of responses may make summarisation and
interpretation difficult.
Potential for moderator bias
Cost (moderator fee, facility rental, recording and transcribing, data
analysis, participant incentives)
Subjects’ conformity
Designing and Conducting Focus
Groups
“The experience of using the focus group as
a qualitative research method can be
compared with that of the tightrope walker:
when things go well there is a feeling of
exhilaration, when they go badly….it’s a
long drop!”
Pugsley 1996:126
Interviews and Focus Groups
Focus groups are not appropriate when:
1. Detailed probing of an individual’s behaviour, attitudes or needs is
required
2. The subject matter under discussion is likely to be of a highly
confidential nature
3. The subject matter is of an emotionally charged or embarrassing
nature
4. Certain strong, socially acceptable norms exist and the need to
conform in a group discussion may influence response
5. A highly detailed (step-by-step) understanding of complicated
behaviour or decision-making patterns is required
6. The interviews are with professional people or with people on the
subject of their jobs.(Hawkins et al, 1994; 554-444).
Steps in Design and Use of Focus
Groups
Problem definition/formulation of research
question
Identification of sampling frame
Identification of moderator
Generation and pre-testing of interview guide
Recruiting the sample
Conducting the group
Analysis and interpretation of data
Writing report
Designing Focus Groups: How do you
recruit participants?
Time-consuming
Krueger (1988: 94) refers to ‘recruiting on
location’
Morgan (1995) says recruitment is the single
most common source of failure he has
encountered in focus group research.
How to avoid problems: repeated contacts,
offering incentives, over-recruiting. (Morgan
1988 suggests over-recruiting by 20%).
May be recruited by existing social networks,
word of mouth or advertising.
Designing Focus Groups: How many people in a group?
Literature differs but researchers highlight that size should be
related to research topic/purpose
Group sizes of 4 to 12 are recommended with an ideal group the
size of 8 (Morrison & Peoples, 1999; Diloria 1994 et al.)
6 to 9 (Garrison et al. 1999)
Generally 8-12 individuals (Stewart and Shamdasani 1990)
6 to 10 (MacIntosh 1993)
Up to 15 (Goss and Leinbach 1996)
Smaller groups may be dominated by one or two members
Larger groups may be difficult to manage, obtain the perspectives
of all members
Designing Focus Groups: Who should make
up your focus groups? (Sampling as Part 2)
The issue is sample bias not generalisability (Morgan
1988)
Typically use purposefully selected samples.
Heterogeneous or homogenous.
Do you want to make comparisons between different
groups?
Key question: Would these groups normally discuss the
topic in day-to-day interaction? (Morgan 1988).
Designing Focus Groups: How many
groups?
Depends on approach; research questions; time and budget
constraints
Some use only one meeting with each of several focus groups (e.g.
Burgess 1996)
Others use follow-up meetings (e.g. Pini 2002)
Multiple groups of similar participants are usually necessary for data
to be valid
Most questions can be answered by 6 to 8 groups, although 4
groups are adequate for some studies and 50 are needed for more
extensive studies (Krueger 1994)
One important determinant is the number of different population subgroups required (Morgan 1988)
Designing Focus Groups: How long
should they last?
One and a half to two and a half hours (Stewart and
Shamdasani 1990)
Consider moderator as well as participant
fatigue. Keim et al (1999) study used one hour groups
for children and found this was too long.
Designing Focus Groups: Developing a
focus group guide
In general, keep the number of broad concepts
examined in a focus group moderate so that
each can be examined in detail.
Tend to be general in nature and open-ended.
Moderator will be improvising comments and
questions within the framework.
Opening question is one that everyone answers
at the beginning of the focus group.
Designing Focus Groups: What is the
role of the moderator?
Smith (1995) recommends two moderators for
better control of group cohesion and more
thorough observation of group dynamics.
Morgan (1988: 49) favours approach he calls
‘highly nondirective focus groups’ or what he
says are ‘self-managed groups’.
Moderator needs to have both strong
interviewing and observational skills (McDonald
1993).
What is the role of the moderator?
Consider advantages of high moderator involvement:
Can cut off unproductive discussion
Ability to ensure all topics covered
Can adjust discussion
Consider disadvantages of high moderator involvement
A biased moderator will produce data that reproduces these
biases
Does not allow new / unanticipated issues to emerge
Consider advantages of low moderator involvement
Can assess participants’ own interests
Participants can bring up controversial topics/topics not
considered by moderator
Consider disadvantages of low moderator involvement
Relatively disorganised in content and so more difficult to
analyse
Some topics may never come up
Analysing Focus Group Data
A typical two hour session yields an average of 40 to 50
transcript pages.
Morgan (1988: 64): The group is the fundamental unit of
analysis and the analysis should begin in a group-bygroup progression.
Hyden and Bulow (2003) stress the need to examine not
only pure content, but who is saying what
Krueger (1993) says read transcripts/summaries and:
Consider the words
Consider the context
Consider the internal consistency
Consider the specificity of responses
Find the big ideas
Consider the purpose of the research
Krueger (1993) Quality control in focus group
research.
.
In D. L. Morgan, Successful Focus Groups, Sage, Newbury Park, 65-85
Ten quality factors in focus group research:
Clarity of purpose
Appropriate environment
Sufficient resources
Appropriate participants
Skilful moderator
Effective questions
Systematic and verifiable analysis
Appropriate presentation
The future?
Emerging literature on virtual focus groups.
Who and what are being researched in online
focus groups?
Are online groups going to replace traditional
focus groups?
Are respondents who they say they are?
Do respondents in online groups really interact
with each other?
(See Sweet, 2001; Murray 1997; Walston and Lissitz 2000. )
PART SEVEN
Mixed Method Approaches to
Qualitative Research
Integrative Models
Model 1
Quantitative
or qualitative approach is used
independently of the other
Model 2
Qualitative
approach is used to develop quantitative
measurement scales
Model 3
Qualitative
approach is used to interpret quantitative
findings
Model 4
Quantitative
findings
approach is used to interpret qualitative
Specific ‘Mixed-Methods’
Approaches to Research
We
have covered the 4 major
qualitative methods, but some
recognised methods combine these
approaches
Case
study method
Ethnography
Action research
Case Study Method
Distinct from “a case” (the object of study)
Features (Yin 2002)
Single
example of a phenomenon of interest
(organisation, part of an organisation)
May involve more than one ‘case’ (multi-site study)
but comparisons between them are a feature of the
research (separately identifiable)
May also only involve a single case (within-site study)
Used in law (illustrative cases), health (unusual or
interesting illnesses), psychology (Freud), political
science (case reports) and business (organisations
with defined features)
Case Study Method (cont.)
May be qualitative or quantitative or both,
but relies on multiple sources of evidence
where data triangulates in a converging
fashion
For qualitative case studies, Yin (1989)
suggests 6 types of information:
Observations,
interviews, audio-visual
material, documents, archival material,
physical artifacts
Case Study Method (cont.)
Challenges in case study method:
number of cases selected – the more cases, the
more diluted the overall analysis. Typically no more
than 4
The issue of Single case study research
Choosing the case(s) – strong rationale for purposeful
sampling strategy is important
Deciding the ‘boundaries’ of a case – how it might be
constrained in terms of time, events and processes
Presents general propositions but not broadly
generalisable, but should it be?
The
Debates on Generalisability
Critique of the importance and goal of
generalisability (the discovery of laws, Lincoln &
Guba)
Attribute the belief to positivism
Critique the view that we can produce
knowledge that
is free of time and context
Argue that the choice is not about searching for
general laws OR studying the unique, but something
in between
i.e. stating conclusions from studying one context that
might hold in another context, ‘working hypotheses’,
the ‘fit’ between one case study and another,
generalising not about what is, but what may be or
what could be
Ethnography
Genesis in cultural anthropology
Argued to be not one particular method but a style of
research that is distinguished by its objectives
Some overlap with participant observation as this is the
predominant technique used. However, interviews and
documentary methods also often utilised.
Definition:
To understand social meanings and activities of people in a
given setting
“a description and interpretation of a cultural or social group or
system” (Cresswell 1998)
Mostly used in anthropology and sociology, but also
health sciences, education, rarely in business
Ethnography (cont.)
Has a number of features distinct from other
methods:
Sees
the world through the eyes of those being
researched, allowing them to speak for themselves
Researchers immerse themselves in the setting and
become part of the group in which they are interested
Aims to provide understanding of the meaning and
importance that members of the group impart to their
own behaviour and that of others
Ethnography (cont.)
Key terms
Fieldwork
– collecting data in a particular
setting
Gatekeepers
Key informants
Reciprocity
Reactivity or reflexivity
Action Research – a ‘participatory
approach to enquiry’
Definition:
“Disciplined enquiry (research) which seeks focused efforts to
improve the quality of people’s organisational, community and
family lives” (Calhoun 1993).
Key tenets:
Processes are rigorously empirical and reflective (research is
self-conscious)
Research engages people who have traditionally been called
“subjects” as active participants in the research process
Research results in some practical outcome related to the lives
or work of the participants
Democratic, equitable, liberating, life enhancing
Operates at intellectual, as well as social, cultural, political and
emotional levels
Action Research (cont.)
Has much in common with community
development and practitioner research
Routine is look, think, act… or observation,
reflection, action…
However, not linear, neat or orderly, rather
routine can work backwards, in repetition and
revision, can leap frog stages and sometimes
make radical changes in direction
PART EIGHT
Qualitative Analysis
Qualitative Analysis
“If the sociologist or the biographer is like a detective,
and collecting data is like detection, then analysing data
is akin to the culminating stages of the criminal justice
process. It has the same potential for abuse, and
therefore requires similar safeguards. Unfortunately,
whereas in criminal justice the adversarial roles of
prosecution and defence can be allocated to different
people, in qualitative analysis the analyst often has to
play both roles”
Dey, 1993
Your approach to qualitative data analysis will be informed by your
epistemological stance (Chua, 1986)
Three major philosophical positions in qualitative
analysis:
1. Positivist
Evidence of formal propositions, quantifiable measures of
variables, drawing of inferences about a phenomenon from a
representative sample to a stated population
Eg Content analysis – simply counting words / phrases (e.g.,
political speeches; media articles)
Relational content analysis – more in-depth; considers meanings
of excerpts and the relationship between them
2. Critical
Main task is social critique; helps to eliminate the causes of
unwarranted alientation
Approaches to qualitative data analysis
3. Interpretive
Knowledge is gained through social constructions such as
language, consciousness, shared meanings etc. Does not
predefine dependent and independent variables; seeks to
understand phenomena through the meanings that people assign
to them
E.g. Grounded theory – inductive, theory-building approach
Narrative analysis – preserves the story rather than fragmenting
data
Phenomenological approaches – particularly concerned with
generating meanings and gaining insights into phenomena
Discourse analysis – assumptions and meanings underlying
spoken language, ‘main line story’
Conversation analysis – highly specialised, based on linguistics
(Atkinson & Delamont argue divorced from wider issues such as
identity, interactions and social encounters)
Interpretivist Approaches
“Interpretivist positions are concerned with how
the social world is interpreted, understood,
experienced, produced or constituted. While
different versions of qualitative research might
understand or approach these elements in
different ways (e.g. focus on social means, or
interpretations, or practices, or discourses, or
processes, or constructions), all will see at least
some of these as meaningful elements in a
complex – possibly multi-layered and textured –
social world”
Mason 2002
How to ‘Read’ Data
1. Literally
actual words and language used – the literal
content of the data
The sequence of interaction – in the case of
interviews, who speaks when?
In the case of visual data – style, layout, literal form
Although these categories may be important, few
researchers will stop here. Some argue that purely
objective description is not possible because the
social world is always interpreted and what we see is
shaped by how we see it!
The
How to ‘Read’ Data
2. Interpretively
Constructing
or documenting a version of
what you think the data mean or represent
Reading through or beyond the data
E.g. implicit norms or rules with which an
interviewee is operating
Discourses that influence people
Versions or accounts of how people make
sense of social phenomena
How to ‘Read’ Data
3. Reflexively
Locates
the researcher as part of the data
generated
Seeks to explore the role and perspective of
the researcher in the process of generation
and interpretation of data
Captures or expresses the relationships
between researcher and data
E.g. response to a certain situation in
fieldnotes (empathy, shock, agreement,
amusement)
Stages in the Analysis of
Qualitative Data
Stage 1: Immersion
The researcher intensively reads or listens to material, assimilating as
much of the explicit and implicit meaning as possible
Stage 2: Categorisation
Systematically working through the data, assigning coding categories or
identifying meanings within the various segments / units of the ’text’
Stage 3: Reduction
questioning or interrogating the meanings or categories that have been
developed? Are there other ways of looking at the data? Do some
codes mean the same thing?
Stage 4: Triangulation
sorting through the categories. Deciding which categories are recurring
and central and which are less significant or are invalid or mistaken
Stage 5: Interpretation
making sense of the data from a wider perspective. Constructing a
model or using an established theory to explicate the findings of the
study
Making a Convincing Argument about your Data (Mason,
2002)
Making a convincing argument will be influenced by the
research questions you originally posed, the focus of the
research and the kinds of data generated
Major categories of arguments:
1. Arguments about how something has developed – a
meaningful process of development or a story or an
‘archaeology’ 2. Arguments about how something works or is
constituted – how and why social phenomena work (but not
cause and effect)
3. Arguments about how social phenomena compare –
meaningful points of comparison in different contexts
4. Arguments about causation and prediction – the effects of
variables on each other; not widely used by qualitative
researchers
Techniques to Ensure Qualitative Data is Credible (Cresswell, 1998)
Triangulation – checking one source of data against
another
Leaving an audit trail – clear records about how the
analysis was conducted
Member checking – have more than 1 researcher
conduct analysis and compare interpretations
Checking for researcher effects – do results differ across
researchers (e.g. focus groups with managers)
Checking the meaning of outliers – find explanations for
‘extreme cases’
Searching for contradictory or negative evidence
Replicating your findings (more difficult in qual research)
Getting feedback from participants
Seeking feedback from peers
Then and Now
Coding historically done by hand – marker
pens, cutting and pasting (scissors and
glue), sorting and shuffling file cards
Early to mid 1980s marked the emergence
of basic data programs for storing and
accessing text
Now at least 25 different programs – some
specifically QDA, some more general
purpose
What software can and cannot do
Software is a tool to help analyse qualitative data. It can:
Store transcripts / other text
Store codes
Search and retrieve segments of text
Link data segments to each other, forming categories, clusters or
networks of information
Make notes
Edit
Conduct content analysis (count frequencies, sequences or
location of words)
Graphically map concepts
It cannot read the text and decide what it means
Similarly, it cannot substitute for learning data analysis
methods
Advantages of Software for Qualitative
Data Analysis
Consistency
all
the places where a code or combination of codes
applied, therefore not missing data that contradicts
incorrect hypothesis
Speed
Once
program is learned and data is set up, much
quicker than manual coding (particularly re-sorting,
re-defining codes and creating matrices of codes)
Graphic maps
Helps
visualising and therefore thinking and
theorising about possibilities and alternatives
Advantages of Software for Qualitative
Data Analysis
Disadvantages
Speed and ease of use can make us lazy
Autocoding (searching key words) may encourage shortcuts
May encourage ‘quick and dirty’ research with premature theoretical
closure
Direct representation of hierarchical relationships (as opposed to
‘circular loops or unstructured networks) encourages hierarchical
thinking
May tempt researchers to skip over the process of ‘proper’ learning
NB As with every methodological decision, if you DO decide to use
software you should justify it in terms of the literature, acknowledge
its limitations (again referring to the literature) etc.
PART NINE
Trustworthiness in Qualitative
Research
Definitions
Reliability – generally understood to concern the
replicability of research findings and whether or not they
would be repeated if another study, using the same or
similar methods, was undertaken.
Validity – traditionally understood to refer to the
‘correctness’ or ‘precision’ of research.
Internal validity: ‘investigating what you claim to be investigating’
(Arksey and Knight 1999)
External validity: ‘the abstract constructs or postulates generated,
refined or tested’ are applicable to other groups within the
population (LeCompte and Goetz 1982)
Examining the terms: validity and reliability
Validity, reliability and
generalisability have been called
the ‘holy trinity’ of the sciences
(Kvale 1996).
What assumptions are inherent in
emphasising the importance of
validity and reliability?
To use or not use the terms validity/reliability
Some qualitative researchers (e.g. critical theorists, feminist
theorists, post-structural theorists) have criticised the use of these
terms in qualitative research. On what basis?
A range of qualitative researchers have denied the relevance of
validity/reliability to qualitative research and argued that qualitative
research has its own procedures and processes for judging and
attaining validity/reliability.
There is still ongoing debate about this in the literature.
However, many qualitative researchers utilise the terms validity and
reliability, and describe a range of strategies to enhance both in their
work.
Your view about reliability/validity will depend upon your own
epistemological, theoretical and methodological position.
Examples of moving away from the terms
validity/reliability
Smith (1996) argues that internal coherence
(or lack of it) would be the most appropriate
way of assessing qualitative research.
Rather than being concerned, for example,
with the representativeness of the sample,
you should concentrate on whether it was
internally consistent and coherent.
Does
it present a coherent argument? Does it
deal with loose ends and possible contradictions
in data? Are the interpretations that the
researcher makes warranted by the data
presented? Does the report deal with alternative
readings?
Leininger 1994
Credibility
Confirmability
Using repeated experiences, events to identify patterns etc
Saturation
Understanding data within holistic contexts (participants’ contexts)
Recurrent patterning
Repeated direct participatory and documented evidence observed or obtained
from primary sources
Meaning-in-context
Ensuring that the researcher uses active listening, reflection and empathic
understanding to grasp what is ‘true’ to informants in their lived environment
Full immersion by the researcher in the phenomena being studied; getting ‘thick’
data to know fully what is being studied
Transferability
Examining general similarities of findings in similar environmental situations
Popay et al 1998
The privileging of ‘subjective meaning’ – the research illuminates the
subjective meaning, actions, and context of those being researched.
Responsiveness to social context – the research design is
adaptable/responsive to real-life situations.
Purposive sampling – the sample produces the knowledge necessary to
understand participants’ location in structures and processes.
Adequate description – the reader can interpret the meaning and context of
what is researched.
Data quality – different sources of knowledge about the same issues are
compared.
Theoretical and conceptual adequacy – the research describes the process
of moving from the data to intepretation.
Typicality – claims are made for logical rather than probabilistic
generalisations.
Lincoln and Guba (1990)
credibility
applicability
consistency
neutrality
Strategies for Promoting Validity and Reliability in
Qualitative Research (Merriam 2002).
Triangulation
Denzin (1970) identifies four types of triangulation to confirm
emerging findings:
Multiple investigators,
Multiple sources of data (time, space, person)
Multiple data collection methods to confirm emerging findings
(observations, interviews, focus groups etc).
Multiple theoretical perspectives. This involves using several
perspectives to examine the same set of data. Few
investigators use this technique.
Janesick (1994: 214) adds interdisciplinary triangulation.
The greater the convergence attained through triangulation the
greater confidence in findings.
First argued by Foreman (1948) ‘to establish validity through
pooled judgement’.
Strategies for Promoting Validity and Reliability in
Qualitative Research (Merriam 2002).
Member checks
Taking
data and tentative interpretations back to the
people from whom they were derived and asking if
they were plausible.
Also called member tests of validity and host
verification
Can be conducted continuously and both formally and
informally (e.g. at the end of an interview by
summarising data and allowing respondent to
immediately correct errors of fact, by giving copies of
various parts of your report to different stakeholders
and asking for comment etc).
Not necessarily free of bias or problems (see St.
Pierre 1999 and Sandelowski 1993 ).
Strategies for Promoting Validity and Reliability in
Qualitative Research (Merriam 2002).
Peer review/examination
Discussions
with colleagues regarding the
process of study, the congruency of emerging
findings with the raw data, and tentative
interpretations.
This peer is outside the context but has some
general understanding of the study and
methods etc.
Criticised by some e.g. Morse (1994)
Strategies for Promoting Validity and Reliability in
Qualitative Research (Merriam 2002).
Researcher’s position/reflexivity
Critical self-reflection by the researcher
regarding assumptions, worldview, biases,
theoretical orientation, and relationship to the
study that may affect the investigation.
Keeping a journal/type of diary.
Includes information about yourself as a
researcher, your schedule, insights, decisions
and justifications for decisions
Strategies for Promoting Validity and Reliability
in Qualitative Research (Merriam 2002).
Adequate engagement in data collection
Adequate time spent collecting data so that data
become ‘saturated’.
This may involve seeking discrepant or negative
cases. Negative case analysis (or analytic
induction) involves addressing and considering
alternative interpretations of the data, particularly
noting pieces of data that would tend to refute the
researcher’s reconstructions of reality.
Prolonged engagement will build trust and
develop rapport and the impact of your presence
may diminish.
Strategies for Promoting Validity and Reliability in
Qualitative Research (Merriam 2002).
Attention to sampling
Maximum
variation: Purposefully seeking
variation or diversity in sample selection to
allow for a greater range of application of the
findings.
Do not suppress/ignore the deviant/the
different.
Allows for and opens up a range of realities
and perspectives.
Strategies for Promoting Validity and Reliability in
Qualitative Research (Merriam 2002).
Audit trail
A detailed account of the methods, procedures and decision
points in carrying out the study.
Lincoln and Guba (1985) give six categories of audit trail
materials:
(1) raw data (interview guides, notes, documents)
(2) data reduction and analysis products
(3) data reconstruction and synthesis products (e.g. data analysis
sheets)
(4) process notes (journal)
(5) materials relating to intentions and dispositions (inquiry proposal,
journal, peer debriefing notes) (6) information relevant to any
instrument development.
Strategies for Promoting Validity and Reliability in
Qualitative Research Cont. (Merriam 2002).
Rick, thick descriptions
Providing
enough description to contextualise
the study such that readers will be able to
determine the extent to which their situation
matches the research context, and hence,
whether the findings can be transferred.
Lincoln and Guba (1985, 125) state that ‘The
description must specify everything that a
reader may need to know in order to
understand the findings (findings are NOT
part of the thick description).
PART TEN
Ethics and Qualitative
Research
Principles of Research Involving Human Subjects
1. Respect for persons
2. Beneficence
treating others as autonomous agents having rights and freedom
not a means to an end
free, voluntary and informed consent privacy and confidentiality
research should be for the good of the subject either directly or
indirectly through benefiting society
possible benefits are maximised and risks minimised
impasse often develops between social good and individual
rights
3. Justice
benefits and harms are to be distributed fairly
vulnerable groups such as cognitively impaired and mentally ill,
their above average rates of institutionalisation and their
dependency on others, have made them a convenient subject
pool for research
who should participate in research poses significant challenges
to policy formation
Key Ethical Concepts
Protection of participant
Informed consent
Use of deception
Debriefing participants
Right to withdraw
Privacy and confidentiality
Protection of Participants
1. Ensure minimal risk
must apply the cost-benefit-ratio
risks unlikely to be greater than any
encountered in
normal lifestyle
must minimise negative outcomes
2. Strategies
obtain advice from professionals
screen vulnerable participants
monitor unforeseen negative events
debrief participants about research
conduct long-term follow-ups
have
counselling or support available
Informed Consent
Rests on 4 elements
competence,
information, understanding of that
information and voluntariness
but… cannot be established in many important areas
of research e.g. critically ill, demented, minors
Social contract
rests
must
on a mutually agreed contract
reveal all aspects that might influence the
decision to participate
Strategies
inform of the general aims of the
associated costs and benefits
consent forms
project
Use of Deception
Subjects are not given an opportunity to provide their informed consent to
participation before data collection.
Examples include covert observation or subject knows they are participating
in research but not the nature of the research.
Problems
deprives participant of the right to informed consent
but… providing all information is likely to influence behaviour and
therefore results
should be avoided if possible
Guidelines governing deception in research:
no more than minimal risk to subjects
rights and welfare of the subjects will not be affected
research cannot practicably be carried out without the deception
where appropriate, subjects are provided pertinent information
about the research after participation (debriefing)
Debriefing Participants
Rationale
traditional
solution to deception problems
participation considered an educational experience
Strategies
give
all information needed and requested
discuss their experience of the research
provide contact details
Right to Withdraw
Rights
can
withdraw consent without any penalty
can request data be destroyed
Controversy
use
of captive audiences (e.g. students,
military, prisoners, employees)
use of incentives
Privacy & Confidentiality
Avoid the use of sensitive questions
Do not record names if possible
Code questionnaires
Warn prior to data collection what identifying information
will be kept
Explain confidentiality procedures
Research ethics and the Internet
Dilemma based on three facts:
Informed consent is not required for data to be collected from the public
domain (naturalistic observation).
The internet is a public domain
Many online communications (email; discussion groups, chat rooms,
newsgroups etc) cultivates an expectation of privacy
The ease and attractiveness of Internet research renders the
medium vulnerable to misuse.
Guidelines:
When subjects are recruited online, need secure server, secure
protection of information during the study and removal of the records
upon study completion
When using data from online discussion groups, removal of any
references to identity, web site or group, location and time of post is
necessary for confidentiality
Research with vulnerable populations –
An EXTENSIVE methodological literature exists on
undertaking research with specific populations
and the ethics and practice of research with
these populations. For example:
Indigenous people
Youth
People with disabilities
Migrants
The aged