Transcript Focus Groups - Midwest Archives Conference

Ciaran B. Trace
MAC 2010 Symposium: Archival User Studies
Dayton, Ohio, October 21-23, 2010






Introduction to focus
groups
Before you begin
Carrying out the focus
group
Data analysis
Sum up
Questions?
Richard A. Krueger and Mary Anne Casey, Focus
Groups: A Practical Guide for Applied Research, Sage:
Los Angeles, 2008.
 David W. Stewart, Prem N. Shamdasani, Dennis W.
Rook, Focus Groups: Theory and Practice, Sage:
Thousand Oaks, 2007.
 Denise Troll Covey, Usage and Usability Assessment:
Library Practices and Concerns Washington, D.C.:
Digital Library Federation Council on Library and
Information Resources, January 2002. Available at
http://www.clir.org/pubs/reports/pub105/pub105.pdf)
 Daniel L. Morgan, Focus Groups as Qualitative
Research, Sage: Thousand Oaks, 1997.


“A focus group is an exploratory, guided interview or
interactive conversation among seven to ten
participants with common interests or characteristics.
The purpose of a focus group is to test hypotheses;
reveal what beliefs the group holds about a particular
product, service, or opportunity and why; or to
uncover detailed information about complex issues or
behaviors from the group’s perspective. Focus group
studies entail several such group conversations to
identify trends and patterns in perception across
groups. Careful analysis of the discussions reveals
insights into how each group perceives the topic of
discussion.” [Covey 2002]

Notion of an interactive conversation among a small
number of participants
 Not focused on the experiences and opinions of any one
participant [would need to do interviews for this]
 Evidence of the differences and similarities in the group
More than one focus group held in order to properly
explore trends and patterns in perception across
groups (each focus group will last anywhere between
one and two hours)
 Need participants and also a moderator and an
observer
 Need to record data and subsequently analyze that
data





Determining patron satisfaction with
resources and services
Helping to shape the future direction of an
archive as part of the strategic planning
process
Developing and evaluating web and catalog
interfaces
Developing and evaluating user education or
reference services

A form of qualitative research method (with all that
that entails)
 Moderator has a distinctive role
 Interaction occurs in a group setting
 There isn’t a very high degree of naturalism
Generally, you are recording not what people do,
prefer etc….. But what they say they do, say they
prefer, etc… Data is verbal and self-reported
 Small number of participants, selection criteria, and
frequent use of convenience sampling mean that you
cannot generalize the results
 The results are generally intelligible and accessible to
all audiences


Can be used as a self-contained method, as an initial/
supplementary source of data, or as part of a multimethod approach (paired with surveys, individual
interviews, participant observation etc)
 Paired with surveys – focus groups as a means to construct
questionnaires (domains, dimensions, wording)
 Paired with individual interviews – focus groups as a means for
devising the interview schedule for unfamiliar topics and vice
versa
 Paired with participant observation – focus groups as a tool to
help you chose among potential field sites
 As you can see…. focus groups can be used as a means to
initially identify appropriate language, questions, or tasks, and
as a follow-up tool to get clarification

Articulate the purpose of the focus group
 Why is there a need for a study such as this?
 What is the problem that the study seeks to
address?
 What kinds of information do you want to get
from the study?
 What wants the information?
 How will the information be used?

Figure out what types of people can give you
the information that you want
 Who has the greatest amount of insight on
this issue?
 What particular characteristics should these
people have?

Figure you how many focus groups you need to
hold and how they should be sequenced
 Usually hold 3 or 4 focus groups – allows you to be
sure that you have reached saturation point and
allows for data to be compared across groups
 If you want to compare and contrast how certain
types of people talk about an issue separate them into
different groups
 If you anticipate that participants will have conflicting
opinions ,hold focus groups in two stages (separate
the groups initially and then bring them together)
 Determine how you will capture the focus
group data
▪ At minimum, will have an observer taking notes
▪ Can also audio tape the focus groups (transcribe
partially or in toto?)
▪ Can also video tape the focus groups (transcribe partially
or in toto?)
▪ Video taping is rarely done – it is considered too
intrusive
▪ Can take a combined approach – notes and audio, for
example
 Think about the ethical issues and determine if
you need to obtain human subjects clearance…
▪ Nature of the group setting brings up privacy concerns –
limits the kind of topics the researcher can pursue
▪ Will need to get explicit permission from participants to
tape (audio and video)
▪ Decide up front who will be able to listen to the audio
tapes (should limit access to research staff)
▪ Make decisions about how the data will be used so that
this information can be conveyed to the participants
 Using the criteria you have already identified, recruit and schedule
participants.
▪ Not looking for generalizability but to minimize bias
▪ Not random sampling but theoretically motivated sampling or
segmentation
▪ Remember that focus groups are composed of people who are
similar to each other in a way that matters to the researcher –
homogeneity is important but you need enough diversity to allow
for contrasting opinions
▪ Involves scheduling six to twelve participants in designated
demographic groups
▪ Usually avoid mixing people with different levels of power or
expertise
▪ “Ideally” participants should not know each other
▪ May want to “pre-screen” to make sure the participants have the
requisite characteristics
 Prepare interview questions….
▪ Think about the types of questions you need
to ask ….and how you are going to sequence
the questions
▪ Also think about what makes for good
questions…..
▪ Normally ask about a dozen questions for a
two hour focus group
▪ Remember that the interview guide will have
a direct bearing on the content of the group
discussion

Good questions are…
 Open ended and one dimensional
 Sequenced (e.g. general before specific,
positive before negative etc)
 Evoke conversation and allow for depth (ask
participants to think back, avoid asking them
“why”)
 Engage participants (can go beyond oral
questions and engage people in doing things –
listing, rating, choosing among alternatives)

Good questions…

Use words that participants would use when
talking about the issue, are easy to say and
easy to understand, are short, are clear,
include well thought out directions…..

Types of questions used in focus groups
 1. Opening question – icebreaker, for all
participants, question should be designed
to be easily and quickly answered (30
seconds or less), shouldn’t be used to
highlight differences among the group
(age, occupation, educational level),
generally not included in the data analysis

Types of questions used in focus groups
 2. Introductory questions – open ended
questions that introduce the topic of
discussion, get people thinking about their
connection to the topic
▪ Should be questions that interest all participants
▪ Method - ask people to remember when they first
encountered the organization, service etc and to
describe the experience. Or “what is the first thing that
comes to mind when you hear the phrase….”

Types of questions used in focus groups
 3. Key questions - focus on the key areas of
concerns. Most of the questions fall into
this category.
 4. Ending question(s) - E.g. is there
anything we should have talked about but
didn’t.

Recruit, schedule, and train a moderator and
observer(s) for each focus group
 Skilled moderator is key - develop rapport with the




participant, remain impartial, and keep the discussion
moving and focused on the research objectives
Moderator must have knowledge of the background topic
Moderators must be able to handle the human element…
Moderators and observers must avoid doing anything that
may bias the results of the study
Goal is to encourage conversation among the participants
– NOT between the moderator and the participants
Main task is to observe the session and to take
extensive notes
 Not only capturing what is said but also
documenting nonverbal signs and body language –
tells you a lot about how the group feels about the
topic as well as giving some indication of how many
people hold the same idea
 Notes should therefore include meta-comments,
e.g. “Jim seemed surprised at Angela’s response…”


Handle all logistical issues associated with running
focus groups
 Securing compensation?, scheduling rooms,
ordering food/drink, having copies of focus group
questions for moderator and observer, testing the
audio- or videotape equipment, creating name
placards for all those taking part in the focus group
 On the day - getting the room ready (light and
temperature), setting up and testing equipment,
bringing supplies (handouts, name placards etc)

The actual focus group itself…role of the
moderator
 Welcomes the participants and thanks them for
participating, introduces and explains their role as
moderator and the role of the observer, reiterates the
purpose of the focus group, confirms human subject
requirements (anonymity, taping, how data will be
used etc), briefly describes the ground rules and how
the focus group will be conducted, emphasizes that
the goal of the focus group is not to reach consensus disagreements and discussion are important
North Carolina State University Library
http://search.ebscohost.com.www.lib.ncsu.edu/userstudies/documents/

Thanks for taking time to join us to talk about some library tools, services, and
resources that can help students with their class assignments. We here at the
library are hoping to develop a suite of web tools to assist students with their
learning experience. We really want to make these web pages as useful for
students as possible. That's why we've asked you to come talk with us. You've
probably noticed our microphone. We're going to tape record this session
because we don't want to miss any of your comments. People often say helpful
things in these sessions that we can't write down fast enough plus we don't want
you to feel that you need to talk at a pace we can write. Please feel free to talk
casually and naturally. (Co-moderator) and I will take what we learn from your
tonight and share your ideas with our colleagues working on these web pages.
Again, this is meant to be a relaxed discussion so we will be on a first name basis
for the duration of this session. Each of you as you can see has a name placard in
front of you. Hopefully the names on them are correct. We assure you that your
names will not be linked in any way to any written materials produced from this
focus group.

We're going to ask you look at some content and tools that we are thinking
about including on these web pages. First we will provide a static version and
would like you to rank them by your first impression of their function, and we
will discuss as a group which of the pieces of content seem most interesting
and helpful from your perspective and how you might envision yourself using
each piece. You can even suggest new pieces of content that we might have
not have thought of. After this initial exercise and discussion we will have each
of you revisit each piece of content and explore their functionality as currently
designed using the laptops in front of you. Try and note mentally how using
these tools reinforces or dilutes the power of the tool you had initially
assessed. We would like you to once again rank your top fives content pieces,
singling out one as the most valuable. We will then discuss each other's
rankings and how things may have changed. We feel that these activities and
discussions will help us dig into what you think about tools, services, and
resources that are helpful to you or that you can imagine being helpful to
other students. There are no wrong answers for what we're going to
discuss. We know you might have differing points of view and we value
everyone's ideas so we really want to hear them.
Sometimes the first question is asked round-robin and is a
question that each participant can answer. In this way each
participant responds and gets comfortable talking.
 Subsequent questions are answered less formally, and more
conversationally.
 Moderator clarifies anything that participants do not
understand.
 The moderator may invite further comments to better
understand what the participants are saying, For example,
“Would you explain that further?” or “Please give me an
example.”
 The moderator uses verbal and body language to invite
comments from shy participants and to discourage others
who may be trying to dominate the conversation.

The probe should also be paired with another
important tool…. the pause. Pause used as a way of
eliciting a response from the participants.
 Moderator must be comfortable with these silences.

You might or might not transcribe the focus group
tapes - cost of transcribing may exceed the benefits
of having a full transcription
 At minimum, can use the tapes to annotate the
observer’s notes
 Ideally, at least two people analyze the data—the
moderator and observer—high inter-rater reliability
 Goal is to examine, categorize and make sense of
the data.

 Review – doing a thorough read through of the data –
notes, transcripts, etc
 Conduct content analysis on the data
▪ Initial coding – involves reading the data and assigning labels or
codes to each piece of relevant information
▪ Can be done by hand, using note cards, or using a computer.
▪ Choice of strategies - note all mentions of the code, note whether
each individual mentioned a given code, note whether each
groups discussion mentioned a particular code
▪ Reduce this information to manageable aggregates by creating
memos for key themes that emerge from this initial coding
▪ Test the themes by going back to look at the data

Reporting….
 Which topics to emphasize in the report
▪ How many groups mentioned the topic
▪ How many people within each group mentioned the
topic
▪ Degree of enthusiasm displayed for the topic
▪ Notion of “group-to-group” validation
 Blend of summary of discussions and direct reporting of
comments
ADVANTAGES OF FOCUS
GROUPS



A way to gather insight
into behavior and
experience from the
participants’ perspective
Allows for the gathering of
rich data, in the
participants own words
Produce a concentrated
amount of data on a
precise topic quickly and at
relatively low cost
ADVANTAGES OF FOCUS
GROUPS



Group discussion creates
synergy and facilitates
recall and insight
Direct interaction with
participants allows for
clarification of responses,
probing of responses, and
follow-up questions
Results of focus group
research are user friendly
and easy to understand
DISADVANTAGES OF FOCUS
GROUPS


Quality of the responses
depends on clarity of the
questions, the moderator’s
skills, and the participants’
understanding of the goals
of the study and what is
expected of them
Rely heavily on the
observational skills of the
moderator and observer
DISADVANTAGES OF FOCUS
GROUPS
Quality of the information is
impacted if participants are
unduly influenced by others,
stray off topic, or seek
consensus
 Hawthorne effect - cannot
observe something without
changing it
 Findings have to be
interpreted at the group level
and are not generalizable
