Transcript Slide 1

Elimination of Familial Sex
Offenders Inflate the
Estimated Efficiency of the
MnSOST-R
Jacqueline Waggoner, Ed.D.
Assistant Professor
University of Portland
503.943.8012
[email protected]
J. Waggoner & R. Wollert
Richard Wollert Ph.D.
1220 SW Morrison St. #930
Portland, OR 97205
360.737.7712
[email protected]
www.richardwollert.com
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
1
Purpose
•
This study is a continuation of an
ongoing evaluation of the validity of
the Minnesota Sex Offender
Screening Tool-Revised (MnSOST-R)
– MnSOST-R is an actuarial test for the
prediction of sexual recidivism (ATSR) of
convicted sex offenders
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
2
Major Components of Actuarial Tests
•
Test items
– Examples: Age, number of sex offenses
– Items added to obtain a total test score
•
Experience table
– Consulted to find out what percentage of
those with the offender’s test score have
re-offended in the past. This is the
offender’s estimated recidivism risk.
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
3
Development of MnSOST-R
•
•
Funds provided by the Minnesota
Department of Corrections (DOC),
1991
Has never been published in a peerreviewed journal
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
4
Measurement Theory and Research
Methodology and the MnSOST-R
•
Does not conform to a number of
criteria for evaluation of tests, testing
practices and effects of test use*
– (Revised Standards for Educational and
Psychological Testing)*
*American Educational Research Association (AERA); American Psychological Association
(APA); National Council on Measurement in Education (NCME)
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
5
Effect of Unpublished Status of
MnSOST-R
•
Limits outlets for scholarly review
and criticism of the MnSOST
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
6
The MnSOST is Widely used in Sexually
Violent Predator (SVP) Cases
•
Overview of SVP Laws
– Involve post-prison civil commitment of
“high risk” sex offenders
– SVP laws have been passed by
legislatures in 16 states and the District
of Columbia
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
7
States with SVP Civil Commitment
Laws
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Arizona
California
Florida
Illinois
Iowa
Kansas
Massachusetts
Minnesota
Missouri
New Jersey
North Dakota
South Carolina
Texas (outpatient commitment)
Virginia
Washington
Wisconsin
District of Columbia
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
8
Why Continued Scrutiny of
MnSOST-R is Important
•
•
Poor test development procedures
Serious implications of the test for
justice and public safety in civil
commitment cases
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
9
Development of MnSOST-R
•
Small original sample (256 incarcerated
sex offenders) for a 16-item test
– Enrichment: included 16 recidivists who did not
belong to the sample
– De-selection: took out 113 familial offenders
who belonged in the sample
•
•
Recidivism Rate = 35% - 41% vs. usual
20% – 25%
“Cut Score” > 13 used for recommendation
for commitment
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
10
Validity Indicators
•
•
•
•
•
Sensitivity at cut score of 13 = .15
1-Specificity = .02
Correlation with recidivism = .45
Area under the ROC Curve = .77
Comparative test efficiency
(calculated by Wollert, 2005, using
Bayes’s Theorem)
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
11
Efficiency of MnSOST vs. Other ATSRs For
High Risk Offenders of Different Ages
Sexual Recidivism Estimates For Sex Offenders Of Different Ages With High Actuarial Scores
0.6
0.5
Recidivism Rate
0.4
MnSOST-R
RRASOR
SORAG
Static-99
VRAG
Commit
0.3
0.2
0.1
0
18-24
25-29
30-34
35-39
40-44
45-49
50-59
60-69
Age at Release
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
12
Most Thoroughly Documented
Shortcoming of the MnSOST
•
Serious flaws have been found in all
proposed experience tables
– 1995 SOST (Base Rate = 41%)
– Replaced by MnSOST-R in 1999
• Base Rate for MnSOST-R was 35%
• Same offenders were in both samples
• No explanation for the drop in base rate
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
13
Flaws in 1999 MnSOST-R
•
Test Contents
– Original Experience Table (Base Rate = 35%)
– Estimated Table for samples with 21% Base
Rate
– Estimated Table for samples with 15% Base
Rate
•
On cross-validation the recidivism rate for
those with scores >13 decreased from 88%
to 44% (Wollert, 2002).
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
14
12/03: A Third Version of MnSOST Was
Formulated
•
•
•
Includes almost all of the original sample of
offenders and a new sample of 220
offenders
Classification was much less accurate for
the new sample than it was for the original
sample
Who knows when the next MnSOST will
appear, or why?
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
15
Another Possible Problem That
Has Not Been Studied Involves the
Elimination of Familial Offenders
•
•
The reliability of this procedure has never
been determined
Raises possibility that some or all of the
MnSOST validity estimates are inaccurate
(Wollert, 2003)
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
16
Method for Studying This Issue
•
•
Obtain data from a representative
sample
Re-calculate validity indicators
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
17
Bartosh et al. (2003) Reported Data
for a Representative Sample
•
•
•
N = 186 Sex offenders released from
prison
r with sexual recidivism = .096 (ns)
ROC = .58 (ns)
– This means the test does not
differentiate recidivists from nonrecidivists
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
18
Bartosh Shared Her Frequency Data
with Waggoner and Wollert
•
Results of Waggoner’s and Wollert’s
analysis of Bartosh data
– Sensitivity at cut score of 13 = .14
– 1- Specificity = .067
• Much larger than that for the nonrepresentative sample (i.e., .02)
– Much less efficient than other ATSRs
when used to predict recidivism for those
with high scores
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
19
Sexual Recidivism Estimates for Sex Offenders Who Have High Actuarial Scores But Vary In
Age
0.6
0.5
Recidivism Rate
0.4
MnSOST
MnSOST (B)
RRASOR
0.3
SORAG
Static-99
VRAG
Commit
0.2
0.1
0
18-24
25-29
30-34
35-39
40-44
45-49
50-59
60-69
Age at Release
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
20
Discussion
•
•
Validity indicators reported by MnSOST
test developers are inaccurate and
idiosyncratic as a result of nonrepresentative sampling
A second study on a large representative
sample (N = 468) is currently underway
with Barbaree and Langton to explore this
problem further
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
21
Conclusion
•
Rationale for SEPT was to identify
procedures that enhance test validity
– Test developers need to be aware of and
follow SEPT procedures
– Clinicians need to be aware of the extent
to which the tests they use are based on
SEPT procedures
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
22
References
•
•
•
•
•
•
•
•
•
American Educational Research Association, American Psychological Association, & National Council on Measurement in
Education. (1999). Standards for educational and psychological testing. Washington, DC.
Bartosh, D. L., Garby, T., Lewis, D., Gray, S. (2003). Differences in the predictive validity of actuarial risk assessment in
relation to sex offender type. International Journal of Offender Therapy and Comparative Criminology, 47(4),
422-438.
Doren, D. M., Dow, E. A. (2003). What “shrinkage” of the MnSOST-R? A response to Wollert (2002b). Journal of Threat
Assessment, 2(4), 49-64.
Epperson, D. L., Kaul, J. D. and Huot, S. J. (1995) Predicting risk of recidivism for incarcerated sex offenders: Updated
development on the sex offender screening tool (SOST). Paper presented at the 14th Annual Research and
Treatment Conference of the Association for the Treatment of Sexual Abusers, New Orleans, Louisiana.
Epperson, D. L., Kaul, J. D., & Hesselton, D. (1997) Final report on the development of the Minnesota Sex Offender
Screening Tool-Revised (MnSOST-R). St. Paul, MN: Minnesota Department of Corrections.
Epperson, D. L., Kaul, J. D., Huot, S., Goldman, R., & Alexander, W. (2003, December). Minnesota Sex Offender
Screening Tool-Revised (MnSOST-R): Development, validation, and recommended risk level cut scores. St.
Paul, MN: Minnesota Department of Corrections.
Wollert, R. W. (2002). The importance of cross-validation in actuarial test construction: Shrinkage in the risk estimates for
the Minnesota sex offender screening tool-revised. Journal of Threat Assessment, 2(1), 87-102.
Wollert, R. W. (2003). Additional flaws in the Minnesota Sex Offender Screening Tool-Revised: A response to Doren and
Dow. Journal of Threat Assessment, 2(4), 65-78.
Wollert, R. W. (2005). An application of Bayes’s Theorem to age-wise sexual recidivism rates. Paper presented at the
meeting of the Western Psychological Association, Portland, OR.
J. Waggoner & R. Wollert
2005 Western Psych. Assn. Convention
April 14-17 Portland, OR
23