Housekeeping

  Ethics Midterm marking 1

QUALITATIVE ANALYSIS CODING (INTRO EXERCISES) QUALITY AND QUALITATIVE ANALYSIS AFFINITY DIAGRAMS

THE QUALITATIVE ANALYTICAL PROCESS

Qualitative Inquiry - Challenge

To make sense of massive amounts of data, reduce the volume of information, identify significant patterns and construct a framework for communicating the essence of what the data reveal

2. The Procedures

1 2 3 4 5 6 7 8 9 Coding/indexing Categorisation Abstraction Comparison Dimensionalisation Integration Iteration Refutation (subjecting inferences to scrutiny) Interpretation (grasp of meaning - difficult to describe procedurally)

Exercise 1

   Coding with pre-defined categories Deductive analysis Theory Testing 6

Exercise 2

    Open coding Inductive analysis Exploratory research Theory building research 7

Constant Comparison

  Each time you code data, compare it with other data to which you applied that code Can also compare with cases/examples from outside your data set (other research…)   Similar to clustering algorithms… Similar to affinity diagrams (discussed later) 8

CONCEPT GENERATION & AFFINITY DIAGRAMS

9

The Science and Art of Qualitative Inquiry (Patton, 1988)



The Science

The scientific part is systematic, analytical, rigorous, disciplined, and critical in perspective



The Art

The artistic part is exploring, playful, metaphorical, insightful, and creative

Critical Thinking

‘Critical Thinking calls for a persistent effort to examine any belief or supposed form of knowledge in the light of the evidence that supports it and the further conclusions to which it tends’ (Glaser, 1941)

or more simply!

Critical Thinking means weighting up the arguments and evidence for and against.

Critical Thinking

• Key points (Glaser, 1941): – Persistence: Considering an issue carefully and more than once – Evidence: Evaluating the evidence put forward in support of the belief or viewpoint – Implications: Considering where the belief or viewpoint leads; what conclusions would follow; are these suitable and rational; and if not, should the belief or viewpoint be reconsidered

Guidance for Creative Thinking

1 Be open 2 Generate options 3 Divergence before convergence 4 Use multiple stimuli - triangulate 5 Side track, zig-zag, and circumnavigate 6 Change patterns 7 Make linkages 8 Trust yourself 9 Work and play at it

The Credibility of Qualitative Analysis

1 2 3 Rigorous techniques and methods for gathering high quality data that is carefully analysed, with attention to issues of validity, reliability, and triangulation The credibility of the researcher , which is dependent on training, experience, track record, status, and presentation of self Philosophical belief in the phenomenological paradigm, that is, a fundamental appreciation of naturalistic inquiry, qualitative methods, inductive analysis and holistic thinking

A Credible Qualitative Study

The write-up for a credible qualitative study needs to address the following issues: 1 What techniques and methods were used to ensure the integrity, validity, and accuracy of the findings 2 What does the researcher bring to study in terms of qualifications, experience, and perspective 3 What paradigm orientation and assumptions ground the study

Principles of Analysing Qualitative Data

1 2 Proceed systematically and rigorously (minimise human error) Record process, memos, journals, etc.

3 4 5 Focus on responding to research questions Appropriate level of interpretation appropriate for situation Time (process of inquiry and analysis are often simultaneous) 6 Seek to explain or enlighten 7 Evolutionary/emerging

Qualitative Research: Common Features of Analytic Methods (Miles & Huberman,1994)

1 Affixing codes to a set of field notes drawn from data collection 2 Noting reflections or other remarks in margin 3 Sorting or shifting through the materials to identify similar phrases, relationships between themes, distinct differences between subgroups and common sequences

Qualitative Research: Common Features of Analytic Methods (Miles & Huberman,1994)

4 Isolating patterns and processes, commonalties and differences, and taking them out to the field in the next wave of data collection 5 Gradually elaborating a small set of generalisations that cover the consistencies discerned in the data base 6 Confronting those generalisations with a formalised body of knowledge in the from of constructs or theories

Goals:

Interface Design and Usability Engineering Articulate:

•

who users are

•

their key tasks Brainstorm designs Refined designs Completed designs

Methods:

Task centered system design Participatory design

User centered design

Evaluate

Psychology of everyday things User involvement Representation & metaphors

Participatory interaction Task / Cognitive scenario walk-through

Graphical screen design Interface guidelines Style guides

Usability testing Heuristic evaluation

low fidelity prototyping methods high fidelity prototyping methods

Products:

User and task descriptions Throw-away paper prototypes Testable prototypes

Field testing

Alpha/beta systems or complete specification

19

brainstorming

 the point is:  to generate MANY, WIDE-RANGING ideas  nutty and absurd are GOOD. go for the extremes (to get out of the rut)  riff off other’s ideas.

 the point is NOT:  to generate excellent, complete, feasible ideas … pressure stifles  to develop or critique ideas … go wide. deep is for later.

process

1.

prepare a list of topics / questions ahead of time; or in a preliminary brainstorm

2.

facilitator takes team through list of topics switch topic when energy ramps down

3.

notetaker takes notes (very important) 4.

switch roles so everyone can play 5.

ground rules 6.

followup

brainstorming is like popcorn

      

ground rules

Postpone and withhold your judgment of ideas:

never criticize

Encourage wild and exaggerated ideas Quantity counts at this stage, not quality Switch topics when the popcorn slows down Build on the ideas put forward by others Every person and every idea has equal worth Elect a facilitator (calls switches) and a note-

taker

Form groups of 8-10

assign a facilitator, note taker  Problem:  User interface for a car proximity detection system  Brainstorm 3 aspects of the problem: (e.g., current problems, physical form factor, activity metaphor, input techniques, etc.) 

go: 5 minutes

follow up

 collect the notes  go through carefully, with judgment turned on  look for  interesting, surprising ideas that might work  ideas that will combine well  promising directions on which you should brainstorm more  keep your notes. at a later design stage, come back to them and see if anything else has become useful in the meantime.

work consolidation:

abstracting specific insights

 one tool: the

affinity diagram

 can use to “consolidate” insights from collected or generated data. for example:  brainstorming about design problems 

categories of problems

 brainstorming about design ideas 

categories of ideas

 comments from users 

categories of desirable / successful features



Qualitative analysis

 Another way to inductively determine appropriate codes

ITSM Guidelines

EXAMPLE

Methodology (Phase I)

HOT Admin findings (4 papers) • Field study: • Interviews • Participatory observation Guidelines from literature (14 papers) • • • • • Field studies Interviews Questionnaires Prototyping Cognitive • walkthroughs Surveying other literature Set of guidelines (164 guidelines) 28

Methodology (Phase I)

HOT Admin publications (4 papers) Guidelines from other literature (14 papers) Set of guidelines (164 guidelines)

Open Coding Axial Coding

Categorized List of Guidelines

29

High level Category Low level Category Guideline Guideline ID number

30

Methodology (Phase I)

HOT Admin publications (4 papers) Guidelines from other literature (14 papers) Set of guidelines (164 guidelines) Open Coding Axial Coding Card Sorting Guidelines Framework 31

Framework of Guidelines

32

how do you make an affinity diagram?

1.

team writes down all data & insights on post-it notes; be sure you can link the post-it detail back to its source!

2.

3.

4.

5.

stick one post-it on the wall a whiteboard or big sheet of butcher paper is best arrange the other post-its around it, grouping by affinity to each other. iteration will be required .

look at each group and see what it has in common; name and describe each group. • “snapshot” the result for documentation digital photo  your design website or notebook • transfer post-its onto xerox paper, 1 sheet / notes-cluster  scan  website

why does an affinity diagram work?

• • use physical arrangement/proximity to understand connections openness to serendipity low cost to rearrange ideas • •    many variants: arrange along axes rather than by affinity tie causes to effects group evidence under assertions

affinity diagram exercise

 Now take your notes from the earlier brainstorming and create an affinity diagram 

go: 8 minutes

debrief