Using Qualitative Methods for Improving Cardiovascular Care Elizabeth H. Bradley, PhD Professor of Public Health Yale School of Medicine American Heart Association Pre-Conference Workshop May 19, 2010 1

Disclosure

Research used in this presentation were funded in part by the Agency for Healthcare Research and Quality, the National Heart, Lung, and Blood Institute, the Commonwealth Fund, and the Donaghue Medical Research Foundation No conflicts of interest to disclose 2

Objectives of Workshop

Develop competencies to:

Define qualitative methods and know when to use them Critically evaluate the methodology of qualitative studies (study design and sampling, data collection, analysis) 3

Workshop Outline

1:00 -2:00 PM What are qualitative methods and when should we use them?

(Examples of Results from Qualitative Studies) 2:00-2:15 PM 2:15-3:15 PM Break Sampling and Data Collection with Examples from Studies 3:15-3:30 PM 3:30-4:30 PM 4:30-5:00 PM Break Data Analysis with Examples from Studies Standards of Rigor and Addressing Limitations of Qualitative Studies 4

Act

Research cycle we know from quantitative reseach

Pick Topic Focus research objectives Report Results Define study design + sample Analyze Data Collect Data

5

What are qualitative methods?

Qualitative methods are a set of research approaches used with the following objectives: To describe a phenomenon To generate hypotheses To develop grounded theory 6

Quantitative versus qualitative objectives Estimate prevalence/incidence of phenomenon,

y

(rather than

describe

phenomenon) Test (rather than generate ) hypothesis concerning predictors, correlates, or consequences of

y

(for instance:

x



y or y



y 1 )

Test (rather than develop ) a theory 7

Products of qualitative research

Objective Describe phenomenon, y Output Key domains of y Taxonomy of y Generate hypotheses Develop theory Recurrent themes Testable hypotheses Conceptual model/theory (boxes w/ arrows) 8

Describing a phenomenon

Not everything that can be counted counts, and not everything that counts can be counted.

- Einstein 9

When is a phenomenon best described with qualitative methods?

When the phenomenon that is multifaceted and complex to understand and interpret When the interest is not just the objective event but also how the event is experienced When social interaction and context are important Examples?

10

Why is describing phenomenon useful?

Distilling a complex phenomenon to its key dimensions or component parts can help develop taxonomy Taxonomy help us compare apples to apples in evaluation (QI interventions, HMOs, etc.) Lays groundwork for valid measurement (improves instrument design and fielding) 11

Example: Taxonomy for Quality Improvement Efforts

Domain Dimensions Exemplar concepts or range Goals Content Specificity Challenge Sharedness High quality; low cost, market share Very specific to nonspecific Very challenging to “easy” goals Widely shared to poorly shared ________________________________________________________________ Administrative Support Philosophy Resources Innovation first; safety net provider; etc.

Human, capital, technical _________________________________________________________________ PI initiatives Type Style of impl.

Clinical pathway; standing orders; Top down; participatory; blameless _________________________________________________________________

12 Bradley et al., JAMA, 2002

Generating hypotheses

Identify causal links that seem to be at work What causes what? What is the consequence? Under what conditions? Can be from what one observes directly in sequencing or from how participants talk about their experiences Remember that these are

hypotheses

(only) 13

Example: Recurrent themes/hypotheses in D2B improvement 1. Explicit organizational goal-setting 2. Visible senior management support 3. Innovative, standardized protocols 4. Flexibility in implementation 5. Uncompromising clinical leaders 6. Collaborative, interdisciplinary teams 7. Specific data feedback 8. Non-blaming culture  PARADOX

Developing grounded theory

Theory: a set of premises or hypotheses about how the world works; a conceptual model Grounded versus axiomatic theory 15

How is grounded theory useful?

Can be interesting in its own right And sets up hypotheses for testing Identifies x and y variables; direction of effects Identifies mediating effects Guides statistical model Helps avoid “fishing” exercise Helps in making sense of observed results 16

Example: Conceptual model for long-term care use

Predisposing Need Attitudes Toward Services Social Norms About Caregiving Intended Use of Long-term Care Actual Use of Long-term Care Enabling Factors Perceived Control

When to use qualitative methods?

When your research objective matches what qualitative methods can accomplish - Describe a phenomenon - Generate (not test) hypotheses - Develop grounded theory When the phenomenon is complex and difficult to measure with existing quantitative approaches When literature and hypotheses are lacking 18

When not to use qualitative methods

When you really want to know and publish how often something occurs Because you think qualitative methods will be easier, cheaper, or faster Because “there is no literature in the area”  still has to be a topic that requires qualitative inquiry 19

Can mixed methods help?

Mixed methods: The integration of qualitative and quantitative methods to improve understanding Can be simultaneous or sequential (qualitative  quantitative; quantitative  qualitative 20

Positive deviance approach is a mixed methods approach (qualitative leads to quantitative) A “positive deviance” approach 1.

2.

3.

4.

5.

Identify top performing hospitals Study them qualitatively Generate hypotheses about top performance Test hypotheses quantitatively; random sample Disseminate evidence in national campaign 21 Bradley et al., Impl Sci 2009

Summary

Qualitative methods provide an approach to understanding what may not have been previously examined and what defies quantitative measurement Qualitative methods must fit with the research objectives to be valid and useful 22

Break

23

Sampling and data collection

24

Common qualitative study designs

In-depth interviewing Focus groups Participant or non-participant observation Case study; ethnography Hybrids 25

Sampling techniques in quantitative research What are key concepts that govern sampling?

- Representative-ness of population (valid) - Big enough (precise) What are different sampling strategies?

How does one determine sample size?

26

Sampling techniques in qualitative research What are key concepts that govern sampling?

- Participants have the experience under inquiry - Diversity; find all dimensions that might matter - If you heard it once, you have heard it 27

Sampling strategy in qualitative research Purposeful sampling (sometimes called purposive)  Theoretical sampling as the highest standard May have a random component, which is augmented with purposeful selections Inclusion criteria (defining “key informants”) - Must have the experience in question - Must be willing to talk about it Looking for broad representation, but not in the proportions that are in the population 28

Sample size in qualitative research

Concept of “theoretical saturation”  When no new concepts emerge from successive interviews  Judgment call to some extent  Typically small samples 29

Sampling for focus groups

Typically 6-10 participants per group Usually 2 5 groups per “strata” Must share some common experience or trait Homogeneity v heterogeneity issues Avoid power differential within group if you can 30

Principles in data collection for quantitative research

Closed-ended measures No room for interpretation by investigator Consistency and reproducibility is goal Premise that abstract can capture truth (example of “age” or “race”) 31

Principles of data collection for qualitative research

Open-ended questions Investigator interpretation is expected (disclosure) Depth, validity is goal  TRUST IS #1 CONCERN Premise that truth is in rich detail, not abstraction Always need consent (usually oral is fine, per IRB) 32

Spend time establishing trust and safety

Body language, facial expressions Describe goals, consent process, data integrity Do not judge anything, no matter how small, ensure that there are no right or wrong answers Important distinction between professional researcher and colleague or friend 33

Techniques in depth interviewing

Discussion guide versus survey instrument Few questions (5), all open-ended, with probes Open-ended interviewing Be authentically

curious

Practice

passive

listening Be

alert

for jargon, unclear links, new concepts

Delve

into things that do not make sense Keep your own views to yourself,

interrupt judiciously

Practice the

5-second pause;

let silence happen 34

Typical questions

Grand tour question

: Tell me about your experience with… (whatever your inquiry is) Can you tell me more about that? (Think back) What was that like for you? What happened next?

You used the term “physician champion,” can you tell me more about that? What did you mean when you used that term?

35

More approaches to interviewing

Wait the participant out…eyebrow raises, etc.

Seek examples, but be careful how you ask this. It can cause defensiveness; you want vignettes, stories, etc.

You do not want their packaging (abstracting) of concept but rather the rich detail so you can interpret Summary question: is their anything I should have asked you that would help me understand xyz?

36

Special concerns in focus groups: establishing ground rules

1. Expect differences of opinions 2. Interested in positive and negative comments 3. Want to hear from everyone

(“if you are talking a lot, I may asked you to give others a chance, and if you are not talking, I may call on you

”) 4. Speak one at a time 5. Respect confidentiality of members 6. Use first names only (no identifying information) 7. We will stick to time limit (1 hour – 90 minutes) 37

Recording data

Multiple approaches For in-depth interviews or focus groups, audio-taping is often successful (helps reliability) and forgotten by participants soon after you begin Informal note-taker other than the lead interviewer In field work (ethnography, observation studies), one often just has field notes and maybe archival data 38

In-depth interviewing and focus group moderator skills

Strong interviewing skills Keen observational skills Ability to control and guide discussion Ability to suppress own personal views Has and projects authentic respect for participants 39

How to develop skills

Watch someone experienced Do some yourself and have your work critiqued Practice (toss early attempts); you learn as you go and analyze your own work Recognize that it is not for everyone!

40

Summary

Sampling and data collection rules of thumb in qualitative methods are almost the opposite from these rules in quantitative methods Authentic, open curiosity is key element – of any good scientist, especially one in qualitative research Self-awareness and practice improves skills 41

Break

42

Data analysis

43

Open-ended data

Quantitative analysis of open-ended data - Counting frequency of different statements, ideas, etc.

Report: “20% said x is a problem” “Content analysis” Qualitative analysis of open-ended data - Develop concepts and themes and models - Report themes and illustrative quotations Populate the “x-axis” 44

Steps in implementing the qualitative analysis

1. Prepare the data 2. Read the data for general understanding 3. Code the data 4. Integrate the data 5. Develop taxonomy, themes, and theory Bradley et al., HSR 2008 45

Codes for organizing qualitative data Codes are tags or labels for assigning meaning to descriptive information Coding is the process of organizing the data into “chunks” that are alike, moving from words and sentences to “incidents” that depict a particular concept 46

Approaches to developing codes 1. Provisional “start list” of codes 2. Purely inductive, or grounded, codes 3. Between “start list” and inductive approaches to coding 47

Major types of codes

Concepts of importance - Could be key variables emerging - Could be reasons why or how something works Characteristics of participant or setting - Often emerge as correlates or medicating factors Potentially non-causal links among concepts - Explicit or implicit evidence of inferences about how different coded concepts may interrelate 48

Beginning coding

Read the transcripts or notes for overall understanding (best to occur with team) Note in words the key concepts (each separately), write memos for the file as needed, mark up margins of transcripts or use software to note Come together in group to review transcripts line by-line, using a constant comparative method 49

Negotiating codes

The group will not agree and that is the beginning of the analysis process…negotiate, talk out the concepts, fleshing out their properties Develop code list from those meetings Read a few more, same process Refine code list, recoding as needed 50

Interplay of data analysis and data collection These happen simultaneously in qualitative research; data analysis informs data collection Can change the discussion guide Can sample in new ways  theoretical sampling!

51

Saturation

As coding continues, code sheet is becoming solidified with the properties of each code clarified for whole team Final code sheet (once saturated) Now, APPLY the final coding structure 52

Examples from studies

53

Standards of rigor and addressing limitations in qualitative studies

54

Evaluating methods of quantitative research Reliability: degree to which findings can be reproduced if conducted on same population with same protocol, even if on average wrong Validity: degree to which findings are on average “true,” even if imprecise Generalizability: degree to which findings in sample reflect the truth for the population about which you want to make some conclusion 55

Improving reliability in qualitative research

1. Tape record (unless intimidating or non-consented) 2. Use multiple interviewers (unless intimidating) 3. Send transcript to interviewee for checking (debated) 4. Independent professional transcription 5. Check accuracy of transcription against the tape 6. Apply code structure in systematic way 7. Check inter-rater reliability of coding (debated) 8. Retain audit trail to document code development and analytic decisions 56

Improving validity in qualitative research

1. Multiple coders with different backgrounds 2. Multiple analysts with different backgrounds using negotiated consensus methods 3. Multiple sources: interviews, observations, written documents, etc., if possible 4. Feed back results to participants (debated) 57

Improving generalizability (aka “transferability”) of qualitative research

Use theoretical sampling to ensure participants reflect diversity of opinions and experiences Seek disconfirming evidence Be sure to saturate before ending data collection 58

The most common criticism: small sample size so not generalizable

Generalizability is a moot issue with qualitative research because one’s objective is to generate hypotheses, NOT make statistical inferences about a population However, findings from qualitative studies should be applicable (“transferable”) to many as long as the sample is sufficiently diverse and theoretical saturation is achieved 59

Concluding remarks

The research method, qualitative or quantitative, must match the research objective Qualitative work is not fast, easy, or cheap Qualitative studies do have subjective components, BUT…..

60

Concluding remarks

So do quantitative studies! We can get lulled into the false sense of security that quantitative research is more objective because there are numbers and statistics Good qualitative research employs rigorous and specific techniques that can enhance reliability, validity, and transferability of findings 61

Thank you

62

References

Bradley EH, Holmboe E, Mattera J, Roumanis S, Radford MJ, Krumholz HK. A qualitative study of increasing beta-blocker use after myocardial infarction: why do some hospitals succeed? Journal of American Medical Association 2001; 285:2604-2611.

Bradley EH, Curry LA, Webster TR, Mattera JA, Roumanis SA, Radford MJ, McNamara RL, Barton BA, Berg DN, Krumholz HM. Achieving rapid door-to-balloon times: How top hospitals improve complex clinical systems. Circulation 2006; 113:1079-1085.

Bradley EH, McGraw SA, Curry LA, Buckser DA, King KK, Andersen R. Expanding the Andersen model: the role of psychosocial factors in long-term care use. Health Services Research 2002; 37:1221-1242.

Bradley EH, Curry LA, Devers K. Qualitative data analysis for health services research: Developing taxonomy, themes and theory. Health Services Research 2007; 42:1758-1772.

Curry LA, Nembhard IM, Bradley EH. Qualitative and mixed methods provide unique contributions to outcomes research. Circulation 2009;119:1442-1452.

Patton, M (1999). Enhancing the quality and credibility of qualitative analysis. Health Services Research 34(5): 1189-1208.

Patton, M. Q. (2002).

Qualitative Research and Evaluation Methods

, 3d ed. Thousand Oaks, CA: Sage Publications.

Pope, C., S. Ziebland and N. Mays. (2000).Qualitative Research in Health Care. Analysing Qualitative Data. British Medical Journal 320 (7227): 114-6 Popay J, Rogers A, and Williams G. (1998). Rationale and standards for the systematic review of qualitative literature in health services research. Qualitative Health Research 8(3):341-351.

63