Transcript Document

RISK COMMUNICATION
John Monahan, Ph.D.
 Communication of the risk of terrorism
 Communication of medical risks
 Communication of violence risk in
forensic psychiatry and psychology
Communication of the Risk of
Terrorism
Problems with the Homeland
Security Advisory System
• False positives: “Crying wolf”
Problems with the Homeland
Security Advisory System
• False positives: “Crying wolf”
• Vagueness
– No substantive content: risk of what?
– No spatial detail: where is the risk taking place?
– No temporal detail: when is the risk taking
place?
• Politicized rather than transparent
• Not prescriptive: what is a citizen supposed to do
to reduce his or her risk?
Effective April 2011
Lessons from the Communication of Terrorism
Risk for Forensic Psychiatry and Psychology
• False positives are a pervasive problem:
“accountability ping-pong” (Tetlock and Mellers,
in press)
• Be as specific as possible in communicating risk
(e.g., type of violence, victim, time-frame)
• Be as prescriptive as possible: risk management.
Communication of Medical Risks
Better Doctors, Better Patients, Better
Decisions: Envisioning Health Care 2020
Gigerenzer & Gray, Eds (2011)
• Relative risk reduction, e.g.: “Lipitor reduces
the chance of stroke by 48%”
• Absolute risk reduction, e.g.: “Lipitor reduces
the chance of stroke from 28 in 1,000 to 15 in
1,000 (i.e., by 13 in 1,000 people, or 1.3%)”
Misleading Communication of Risk
Gigerenzer, Wegwarth, & Feufel, 341 BMJ 791 (2010)
Conveying relative risks [e.g., a 50% reduction] without baseline risk
[e.g., from 10 in 1,000 to 5 in 1,000] is the first “sin” against
transparent reporting.
The second is mismatched framing—reporting benefits, such as
relative risk reductions, in big numbers and harms, such as absolute
risk increases, in small numbers.
If we use the example of a treatment that reduces the probability of
getting disease A from 10 to five in 1,000, whereas it increases the
risk of disease B from five to 10 in 1,000, authors who use
mismatched framing would report the benefit as a 50% risk reduction
and the harm as an increase of five in 1000; that is, 0.5%.
Medical journals permit mismatched framing. One
in three articles in the BMJ, JAMA, and Lancet
[during 2009]... used mismatched framing when
both benefits and harms were reported.
Moreover, absolute risks [e.g., 5 in 1,000]… were
more often reported for harms than for benefits.
Steps can be taken to improve the transparency of risk
communication.
Firstly, editors should enforce transparent reporting in
journal abstracts: no mismatched framing, no relative
risks without baseline risks, and always give… absolute
numbers.
Secondly, institutions that subscribe to medical journals
could give journal publishers two years to implement
the first measure and, if publishers do not comply,
cancel their subscriptions.
Cochrane Review: Using Alternative Statistical Formats
for Presenting Risks and Risk Reductions (2011)
35 studies; 83 comparisons. Participants: Physicians and patients
•Participants (health professionals and consumers) understood…
frequencies better than probabilities.
•[However,] they perceived interventions to be more effective, and
were more persuaded to prescribe or accept an intervention, when
exposed to relative risk reduction [e.g., a 50% reduction] compared to
absolute risk reduction [e.g., from 10 in 1,000 to 5 in 1,000].
•Overall there were no differences between health professionals and
consumers… This would be of concern because health professionals
play a key role in medical decision making. It also suggests that the
formal education and training of health professionals apparently has no
effect on their handling of statistical information.
Lessons from the Communication of Medical
Risks for Forensic Psychiatry and Psychology
• Use frequencies rather than probabilities to
communicate risk
• Express the likely effects of treatment using
absolute risks rather than relative risks
• Increase the “numeracy”—the ability to
understand numbers—of forensic psychiatrists
and psychologists, and of judges.
Communication of Violence Risk in
Forensic Psychiatry and Psychology
COVR Classification Tree
Total Sample
19% Violent
Prior Arrests
None
9% Violent
Impulsiveness
Low
7% Violent
Impulsiveness
High
21% Violent
Prior Arrests
Non-Violent
20% Violent
Father
Used Drugs
No
17% Violent
Father
Used Drugs
Yes
39% Violent
Prior Arrests
Violent
36% Violent
Violent
Fantasies
No
27% Violent
Violent
Fantasies
Yes
53% Violent
Risk Classes
1
0.9
0.8
0.7
0.6
P 0.5
0.4
0.3
0.2
0.1
0
1
2
3
4
Classes
Probability of Violence
95% Confidence Interval
5
Psychiatry Research (2011), 188, 161-165.
COVR Default Categories
●
Category 1: Very low risk [1%/1 of 100]
●
Category 2: Low risk [8%/8 of 100]
●
Category 3: Average risk [26%/26 of 100]
●
Category 4: High risk [56%/56 of 100]
●
Category 5: Very high risk [76%/76 of 100]
Illustrative User-Defined Categories
● Category 1: Discharge without further
evaluation.
● Category 2: Discharge if a routine evaluation is
negative for violence risk
● Category 3: Do not discharge now; re-evaluate
in 72 hours
● Category 4: Do not discharge without a full
evaluation by Dr. Smith
● Category 5: Consider warning potential victim
and/or petitioning for outpatient commitment.
Violence risk assessment and risk
communication
Slovic, Monahan, & MacGregor,
Law & Human Behavior (2000)
Case Vignette
A patient – Mr. James Jones – has been
evaluated for discharge from an acute civil
mental health facility where he has been treated
for the past several weeks. A psychologist whose
professional opinion you respect has done a
state-of-the-art assessment of Mr. Jones. Among
the conclusions reached in the psychologist’s
assessment is the following:
RANDOMLY, EITHER
Patients similar to Mr. Jones are estimated to
have a 20% probability of committing an act
of violence to others during the first several
months after discharge.
OR
Of every 100 patients similar to Mr. Jones, 20
are estimated to commit an act of violence
to others during the first several months
after discharge.
QUESTION
If you were working as a supervisor at this
mental health facility and received the
psychologist’s report, would you recommend
that Mr. Jones be discharged from the hospital
at the present time?
Results
Risk Communication
Do Not
Discharge (%)
20% probability
21
20 of every 100
41
Why?: “Imaging the Numerator”
Representations of risk in the form of probabilities
(10% of 100 patients) led to relatively benign images
of one person, whereas the “equivalent”
frequentistic representations (10 out of 100
patients) created frightening images of violent
patients (e.g., “Some guy going crazy and killing
someone.”)
Subsequent Research
• Forensic evaluators communicate risk primarily in
categories (high, medium, low), and use categories
most often if risk is perceived as high (Grann & Pallvik,
2002)
• Forensic evaluators prefer categories that are
prescriptive (Heilbrun et al, 2004)
• There is much disagreement among forensic evaluators
as to the quantitative content of the categories used
(high, medium, low) (Hilton et al, 2008)
“Numeracy”
Peters, Västfjäll, Slovic, Mertz, Mazzocco & Dickert,
Numeracy and Decision Making, Psychological Science
Numeracy Scale—11 items
Lipkus et al (2001)
• The chance of getting a viral infection is .0005. Out
of 10,000 people, about how many of them are
expected to get infected?
• In the Acme Publishing Sweepstakes, the chance of
winning a car is 1 in 1,000. What percent of tickets
in the Acme Publishing Sweepstakes win a car?
• Imagine that we roll a fair, six-sided die 1,000
times. Out of 1,000 rolls, how many times do you
think the die would come up even (2, 4, or 6)?
Ss given a case vignette and a risk estimate that
was randomly EITHER
“Of every 100 patients similar to Mr. Jones, 10%
are estimated to commit an act of violence to
others during the first several months after
discharge.” OR
“Of every 100 patients similar to Mr. Jones, 10
are estimated to commit an act of violence to
others during the first several months after
discharge.”
QUESTION: What is the risk that Mr. Jones will
harm someone?(Scale: 1=low risk; 6=high risk)
Fig. 2. Numeracy and percentage (10% of 100) versus frequentistic (10 out of 100)
representations of risk.
Peters E et al. Psychological Science 2006;17:407-413
Copyright © by Association for Psychological Science
Why?: Less Numerate People Can’t “Image
the Numerator” in the Probability Format
Less numerate people may be influenced greatly by the
affective imagery elicited by the frequency format, and
do not have access to this affective imagery when
given the probability format.
Communicating the Risk of Violence
by People with Mental Illness
“The Treatment Advocacy Center estimates people
with untreated schizophrenia and bipolar disorder
commit about 1,000 of the 20,000 homicides per
year nationwide” (NJ Star-Ledger, July 26, 2011).
OR
The annual probability of an American (N=
312,000,000) being killed by a person with untreated
schizophrenia or bipolar disorder is .000003.
Legal Uses of Risk Communication
Research
Civil Commitment Standard in Virginia
Pre-2008: “The person presents an imminent
danger to himself or others as a result of
mental illness.”
2008: “There is a substantial likelihood that, as a
result of mental illness, the person will, in the
near future, cause serious physical harm to
himself or others, as evidenced by recent behavior
causing, attempting, or threatening harm.”
Civil Commitment Standard in Virginia
Pre-2008: “The person presents an imminent
danger to himself or others as a result of
mental illness.”
2008: “There is a substantial likelihood that, as a
result of mental illness, the person will, in the
near future, cause serious physical harm to
himself or others, as evidenced by recent behavior
causing, attempting, or threatening harm.”
Risk Classes
1
0.9
0.8
0.7
0.6
P 0.5
0.4
0.3
0.2
0.1
0
1
2
3
4
Classes
Probability of Violence
95% Confidence Interval
5
Question to Judges
What is the lowest likelihood of violence to
others that you would accept… for
authorizing short-term civil commitment?
Risk Classes
1
0.9
0.8
0.7
0.6
P 0.5
0.4
0.3
0.2
0.1
0
1
2
3
4
Classes
Probability of Violence
95% Confidence Interval
5
Official Training Material,
VA Department of Behavioral Health
Cohen, Bonnie, & Monahan, 2009
“A ‘one-in-four’ estimated risk of serious harm in
the near future is sufficient [for civil commitment],
A ‘substantial likelihood’ is not meant to mean
‘more likely than not’ (51%).”
Conclusions
• Violence risk communication is a vibrant area of
research in forensic psychiatry and psychology
• Forensic evaluators prefer to use categories, but
rarely define those categories
• Probabilities are very easily misunderstood and
mischaracterized—they should be avoided
• The lack of “numeracy” is a serious,
underresearched problem
• The law can benefit from more transparent
violence risk communication.
TACK!
[email protected]