Transcript Document

QUALITATIVE ANALYSIS
CODING (INTRO EXERCISES)
AFFINITY DIAGRAMS
CSCI 4163/6610 - WINTER 2014
2 general research approaches
deductive approach vs inductive approach
2
deductive research approach
top-down approach
THEORY
HYPOTHESIS
OBSERVATION
CONFIRMATION
3
inductive research approach
THEORY
TENTATIVE HYPOTHESIS
PATTERN
OBSERVATION
bottom-up approach
4
deductive or inductive
5
Exercise 1
 Open coding
 Inductive analysis
 Exploratory research
 Theory building research
6
Exercise 2
 Coding with pre-defined categories
 Deductive analysis
 Theory Testing
7
THE QUALITATIVE ANALYTICAL
PROCESS
Agenda
 2 Main QA approaches
 Coding exercise 1
 Coding exercise 2
 Slides on Qualitative Analysis
 Brainstorming Exercise
 Affinity Diagramming Exercise
9
1. Analysis Considerations
1 Words
2 Context (tone and inflection)
3 Internal consistency (opinion shifts during
groups)
4 Frequency and intensity of comments (counting,
content analysis)
5 Specificity
6 Trends/themes
7 Iteration (data collection and analysis is an
iterative process moving back and forth)
2. The Procedures
1 Coding/indexing
2 Categorisation
3 Abstraction
4 Comparison
5 Dimensionalisation (relationships)
6 Integration
7 Iteration
8 Refutation (subjecting inferences to scrutiny)
9 Interpretation (grasp of meaning - difficult to
describe procedurally)
Qualitative Inquiry - Purpose
The purpose of qualitative inquiry is to produce
findings. The Data Collection process is not an
end in itself. The culminating activities of
qualitative inquiry are analysis, interpretation,
and presentation of findings.
Qualitative Inquiry - Challenge
To make sense of massive amounts of data,
reduce the volume of information, identify
significant patterns and construct a framework
for communicating the essence of what the
data reveal
Qualitative Inquiry - Problem
‘…have few agreed-on canons for qualitative
data analysis, in the sense of shared ground
rules for drawing conclusions and verifying
sturdiness’ (Miles and Huberman, 1984)
The Creativity of Qualitative
Inquiry
 ‘..the human element of qualitative inquiry is both
its strength and weakness - its strength is fully
using human insight and experience, its weakness
is being so heavily dependent on the researcher’s
skill, training, intellect, discipline, and creativity.
The researcher is the instrument of qualitative
inquiry, so the quality of the research depends
heavily on the qualities of that human being’
(Patton, 1988)
The Science and Art of Qualitative
Inquiry (Patton, 1988)
 The Science
The scientific part is systematic, analytical,
rigorous, disciplined, and critical in perspective
 The Art
The artistic part is exploring, playful,
metaphorical, insightful, and creative
Critical Thinking
‘Critical Thinking calls for a persistent effort to
examine any belief or supposed form of
knowledge in the light of the evidence that
supports it and the further conclusions to
which it tends’ (Glaser, 1941)
or more simply!
Critical Thinking means weighting up the
arguments and evidence for and against.
Critical Thinking
• Key points (Glaser, 1941):
– Persistence: Considering an issue carefully
and more than once
– Evidence: Evaluating the evidence put
forward in support of the belief or viewpoint
– Implications: Considering where the belief
or viewpoint leads; what conclusions would
follow; are these suitable and rational; and
if not, should the belief or viewpoint be
reconsidered
Guidance for Creative Thinking
1
2
3
4
5
6
7
8
9
Be open
Generate options
Divergence before convergence
Use multiple stimuli - triangulate
Side track, zig-zag, and circumnavigate
Change patterns of thinking
Make linkages
Trust yourself
Work and play at it
The Credibility of Qualitative
Analysis
1 Rigorous techniques and methods for gathering highquality data that is carefully analysed, with attention
to issues of validity, reliability, and triangulation
2 The credibility of the researcher, which is dependent
on training, experience, track record, status, and
presentation of self
3 Philosophical belief in the phenomenological
paradigm, that is, a fundamental appreciation of
naturalistic inquiry, qualitative methods, inductive
analysis and holistic thinking
A Credible Qualitative Study
A credible qualitative study needs to address
the following issues:
1 What techniques and methods were used to
ensure the integrity, validity, and accuracy of
the findings
2 What does the researcher bring to study in
terms of qualifications, experience, and
perspective
3 What paradigm orientation and assumptions
ground the study
Principles of Analysing
Qualitative Data
1 Proceed systematically and rigorously (minimise human
2
3
4
5
6
7
error)
Record process, memos, journals, etc.
Focus on responding to research questions
Appropriate level of interpretation appropriate for
situation
Time (process of inquiry and analysis are often
simultaneous)
Seek to explain or enlighten
Evolutionary/emerging
Qualitative Research: Common Features of
Analytic Methods (Miles & Huberman,1994)
1 Affixing codes to a set of field notes drawn
from data collection
2 Noting reflections or other remarks in margin
3 Sorting or shifting through the materials to
identify similar phrases, relationships
between themes, distinct differences
between subgroups and common sequences
Qualitative Research: Common Features of
Analytic Methods (Miles & Huberman,1994)
4 Isolating patterns and processes,
commonalties and differences, and taking
them out to the filed in the next wave of data
collection
5 Gradually elaborating a small set of
generalisations that cover the consistencies
discerned in the data base
6 Confronting those generalisations with a
formalised body of knowledge in the form of
constructs or theories
Interface Design and Usability Engineering
Goals:
Articulate:
•who users are
•their key tasks
Task
centered
system
design
Methods:
Evaluate
Brainstorm
designs
Psychology of
everyday
things
Participatory
design
User
involvement
Usercentered
design
Representation
& metaphors
Participatory
interaction
Task /
Cognitive
scenario
walk-through
low fidelity
prototyping
methods
Products:
User and
task
descriptions
Throw-away
paper
prototypes
Refined
designs
Graphical
screen
design
Interface
guidelines
Style
guides
Completed
designs
Usability
testing
Field
testing
Heuristic
evaluation
high fidelity
prototyping
methods
Testable
prototypes
Alpha/beta
systems or
complete
specification
25
brainstorming
 the point is:
 to generate MANY, WIDE-RANGING ideas
 nutty and absurd are GOOD. go for the extremes
(to get out of the rut)
 riff off other’s ideas.
 the point is NOT:
 to generate excellent, complete, feasible ideas
… pressure stifles
 to develop or critique ideas
… go wide. deep is for later.
process
1. prepare a list of topics / questions
ahead of time; or in a preliminary brainstorm
2. facilitator takes team through list of topics
switch topic when energy ramps down
3. Note taker takes notes (very important)
4. switch roles so everyone can play
5. ground rules
6. Follow up
brainstorming is like
popcorn
ground rules

Postpone and withhold your judgment of ideas:
never criticize

Encourage wild and exaggerated ideas

Quantity counts at this stage, not quality

Switch topics when the popcorn slows down

Build on the ideas put forward by others

Every person and every idea has equal worth

Elect a facilitator (calls switches) and a notetaker (one thought per post it!)
Form 5 groups
assign a facilitator, note taker
Problem:
How to design the user interface for
a car proximity detection system
 Brainstorm 3 aspects of the
problem: (e.g., physical form
factor, safety issues, input
techniques, etc.)
go: 5 minutes
follow up
 collect the notes
 go through carefully, with judgment turned on
 look for
 interesting, surprising ideas that might work
 ideas that will combine well
 promising directions on which you should
brainstorm more
 keep your notes. at a later design stage, come
back to them and see if anything else has
become useful in the meantime.
work consolidation:
abstracting specific insights
 one tool: the affinity diagram
 can use to “consolidate” insights from collected or
generated data. for example:
 brainstorming about design problems
 categories of problems
 brainstorming about design ideas
 categories of ideas
 comments from users
 categories of desirable / successful features
how do you make an affinity diagram?
1.
team writes down all data & insights on post-it notes;
be sure you can link the post-it detail back to its source!
2.
stick one post-it on the wall
a whiteboard or big sheet of butcher paper is best
3.
arrange the other post-its around it, grouping by affinity
to each other. iteration will be required.
4.
look at each group and see what it has in common;
name and describe each group.
5.
“snapshot” the result for documentation
• digital photo  your design website or notebook
• transfer post-its onto xerox paper, 1 sheet / notes-cluster
 scan  website
why does an affinity diagram work?
•
use physical arrangement/proximity to
understand connections
•
openness to serendipity
•
low cost to rearrange ideas
•
many variants:
 arrange along axes rather than by affinity
 tie causes to effects
 group evidence under assertions
affinity diagram exercise
Now take your notes from the earlier
brainstorming and create an affinity diagram
go: 8 minutes
debrief