Transcript Partially Specified Actuarial Tables and the Poor Performance of the
Partially Specified Actuarial Tables and the Poor Performance of Static-99R
Richard Wollert Ph.D.
Washington State Vancouver [email protected]
http://richardwollert.com
360.737.7712
American Psychology-Law Society March 2013 Portland, OR Jacqueline Waggoner Ed.D.
University of Portland [email protected]
wordpress.up.edu/waggoner 503.943.8012
1
Actuarial Instruments for Sex Offender Risk Assessment
Contain “risk items” correlated with sexual recidivism.
Each risk item is subdivided into categories. An offender is assigned to only 1 category per risk item. American Psychology-Law Society March 2013 Portland, OR
2
Actuarial Manual
Sets forth criteria for assigning offenders to item categories.
Contains coding rules that weight each category.
Some categories scored as zero, some as 1 or more, a few as – 1 or less. American Psychology-Law Society March 2013 Portland, OR
3
Actuarial Manual
An offender is assigned to a “risk group” per his score. Some groups include a range of scores. We call them “bins.” American Psychology-Law Society March 2013 Portland, OR
4
One-Way Model (Once Called “ Partial Specification ” but Dropped as a Misnomer) Tries to capture the effects of risk factors on recidivism with a single number.
First generation actuarials were one-way models.
The 10-item Static-99 is an example.
Offenders got one point for “ current age less than 25.
” No points if older. American Psychology-Law Society March 2013 Portland, OR
5
One-way Actuarial Table for Static-99 Score Bins and Point Scores (from Hanson & Thornton, 2000, p. 129) .
American Psychology-Law Society March 2013 Portland, OR
6
The “Age Invariance Effect” (Hirschi & Gottfredson, 1983) Sexual recidivism declines with age throughout life (Hanson, 2002). The decline is linear. The effect applies to all risk bins (Wollert, 2006; Hanson, 2006).
Static-99 combined bin-wise rates for all ages.
This masked the fact that different age groups have different recidivism rates. American Psychology-Law Society March 2013 Portland, OR
7
Static-99 Underestimated Young Offender Rates (-%) and Overestimated Old Offender Rates (+%) L Bins L ML MH H Even With Optimum (Unweighted) Scaling Age Groups [*=differences that fall below (-) or exceed (+) the .05 CI] 18-29
-2.0% *
-1.7%
-4.9% * -6.4% *
30-39 -1.4% -.6%
-3.9% *
-3.4% 40-49 +.5%
+1.3% * +4.7% *
+1.8% 50-59 +.8% +1.0% +1.6% +5.3% 60-70+
+2.4% * +3.5% *
+8.4%
+15.3% *
American Psychology-Law Society March 2013 Portland, OR
8
The MATS-1 (Wollert et al., 2010) Took Into Account the Linearity of Age Invariance and Addressed the Estimation Errors of Static-99 MATS-1 = “ Multisample Age-Stratified Table of Sexual Recidivism Rates.
” Removes age item from Static-99, so it has 9 “ non-age predictor ” (NAP) items.
Recidivism focus is on an offender ’s age and NAP score (able to capture interactions).
Also called a “ two-way ” model.
American Psychology-Law Society March 2013 Portland, OR
9
Scores Low Medium High All Levels MATS-1 Recidivism Rates 18-39.9 7.6 17.3 36.2 13.2 40-49.9 Age Groups 50-59.9 4.0 8.0 25.5 2.6 6.4 23.2 7.6 5.6 60 and Over 2.0 2.5 6.4 2.7 American Psychology-Law Society March 2013 Portland, OR
10
Static-99R Is A One-Way Model Designed To Account For The Age Effect Described in Helmus et al., 2012.
Age-weighting was used.
18-34 group: One point added.
40-59 group: One point subtracted.
60-70+ group: Three points subtracted.
American Psychology-Law Society March 2013 Portland, OR
11
Static-99R Performed Poorly
Construction sample ROC = .708.
Validation sample ROC = .720.
Static-99 validation sample ROC = .713.
Recidivism rate for the Static-99R high bin < 27%.
American Psychology-Law Society March 2013 Portland, OR
12
How Age-Weighting Undermined Static-99R ’s Performance: Part 1 of a 3 Part Story
243 young offenders were moved to the highest risk bin from lower Static-99 bins because they received an extra point.
This is “ upscale dilution.
” Less dangerous offenders are mixed with more dangerous offenders = high bins have lower rates (Waggoner et al., 2008).
American Psychology-Law Society March 2013 Portland, OR
13
How Static 99R’s Performance Was Undermined by Age-Weighting: Part 2.
230 old offenders were taken out of the high bin and moved to lower bins because they received negative points.
This is “ downscale enrichment.
” More dangerous offenders are mixed with less dangerous offenders = low bins have higher rates.
American Psychology-Law Society March 2013 Portland, OR
14
How Static 99R’s Performance Was Undermined by Age-Weighting: Part 3.
The numbers of recidivists and nonrecidivists in each bin were about the same for Static-99 and Static-99R when offender data were pooled across age groups.
It is impossible to obtain accuracy differences using ROC tests when the binwise distributions of recidivists and nonrecidivists for two tests are about the same.
American Psychology-Law Society March 2013 Portland, OR
15
The Number of Recidivists and Nonrecidivists In Each Static-99 and Static-99R Bin Were Similar Bins L-99 L-99R ML-99 ML-99R MH-99 MH-99R H-99 H-99R Number of Recidivists 98 113 185 166 265 248 308 326 Number of Nonrecidivists 2,282 2,723 2,500 2,043 1,565 1,589 903 898 Recidivism Rate (5-years) .041 (.034-.050) .040 (.033-.048) .069 (.060-.079) .075 (.065-.087) .145 (.129-.162) .135 (.120-.151) .254 (.231-.280) .266 (.242-.292) American Psychology-Law Society March 2013 Portland, OR
16
Bins L ML MH H Static-99R Bins Underestimate Recidivism for Young Offenders and Overestimate It for Old Offenders. Age Groups [*=differences that fall below (-) or exceed (+) the .05 CI] 18-29 30-39 40-49 50-59 60-70+
-2.1% *
-1.1%
-5.9% * -5.2% *
0 -1.5%
-4.9% *
-2.2% +.4%
+1.9% * +3.7% *
+3.0% +.7% +1.6% +.6% +6.5%
+2.3% * +4.1% *
+7.4%
+16.5% *
American Psychology-Law Society March 2013 Portland, OR
17
Discussion Age-weighting did not enhance Static-99R. Like Static-99, it underestimates young offender rates and overestimates old offender rates.
American Psychology-Law Society March 2013 Portland, OR
18
A Solution to Age-Weighting Problems: Convert Static-99R to a 2-Way Model Take all the age points out of Static-99R.
Stratify Static-99R NAP bins by age in one table.
Use external data and frequency or Bayesian math to construct another table like the first. Assign the cells in Table 1 to bins on the basis of the cell-wise recidivism rates in Table 2.
e.g., cells with very large rates in Table 2 make up Table 1 ’ s “ high ” bin category, etc. American Psychology-Law Society March 2013 Portland, OR
19
References
Hanson, R. K. (2002). Recidivism and age.
Journal of Interpersonal Violence
,
17
, 1046-1062. Hanson, R. K. (2006). Does Static-99 predict recidivism among older sexual offenders?
Sexual Abuse: A Journal of Research and Treatment, 18,
343-355.
Hanson, R. K. & Thornton, D. (2000). Improving risk assessments for sex offenders: A comparison of three actuarial scales.
Law and Human Behavior, 24
, 119-136. Helmus, L., Thornton, D., Hanson, R. K., & Babchishin, K. M. (2012). Improving the predictive accuracy of the Static-99 and Static-2002 with older sex offenders: Revised age weights.
Sexual Abuse: A Journal of Research and Treatment
,
24
(1), 64-101. DOI: 10.1177/1079063211409951.
American Psychology-Law Society March 2013 Portland, OR
20
References
Hirschi, T. & Gottfredson, M. (1983). Age and the explanation of crime.
American Journal of Sociology
,
89
, 552-584.
Waggoner, J., Wollert, R., & Cramer, E. (2008). A respecification of Hanson ’s updated Static-99 experience table that controls for the effects of age on sexual recidivism among young offenders.
Law, Probability and Risk, 7,
305-312.
Wollert, R. (2006). Low base rates limit expert certainty when current actuarial tests are used to identify sexually violent predators: An application of Bayes ’s Theorem.
Psychology, Public Policy, and Law
,
12
, 56-85. Wollert, R. (2007, August). Validation of a Bayesian Method for Assessing Sexual Recidivism Risk. Presented in San Francisco at the 2007 APA conference. http://www.richardwollert.com
American Psychology-Law Society March 2013 Portland, OR
21
References
Wollert, R., Cramer, E., Waggoner, J., Skelton, A., & Vess, J. (2010). Recent research (N=9,305) underscores the importance of using age-stratified actuarial tables in sex offender risk assessments.
Sexual Abuse: A Journal of Research and Treatment
,
22
, 471-490. DOI: 10.1177/1079063210384633.
Acknowledgements
The authors are indebted to Brian Abbott, David Cooke, Ted Donaldson, Elliot Cramer, and Diane Lytton for reading and commenting on previous versions of this presentation.
American Psychology-Law Society March 2013 Portland, OR
22