J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 1 Elimination of Familial Sex Offenders Inflate the Estimated Efficiency of the MnSOST-R Jacqueline Waggoner, Ed.D. Richard Wollert Ph.D. Assistant Professor1220 SW Morrison St. #930 University of PortlandPortland, OR

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 2 Purpose This study is a continuation of an ongoing evaluation of the validity of the Minnesota Sex Offender Screening Tool-Revised (MnSOST-R) –MnSOST-R is an actuarial test for the prediction of sexual recidivism (ATSR) of convicted sex offenders

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 3 Major Components of Actuarial Tests Test items –Examples: Age, number of sex offenses –Items added to obtain a total test score Experience table –Consulted to find out what percentage of those with the offender’s test score have re-offended in the past. This is the offender’s estimated recidivism risk.

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 4 Development of MnSOST-R Funds provided by the Minnesota Department of Corrections (DOC), 1991 Has never been published in a peer- reviewed journal

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 5 Measurement Theory and Research Methodology and the MnSOST-R Does not conform to a number of criteria for evaluation of tests, testing practices and effects of test use* – (Revised Standards for Educational and Psychological Testing)* *American Educational Research Association (AERA); American Psychological Association (APA); National Council on Measurement in Education (NCME)

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 6 Effect of Unpublished Status of MnSOST-R Limits outlets for scholarly review and criticism of the MnSOST

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 7 The MnSOST is Widely used in Sexually Violent Predator (SVP) Cases Overview of SVP Laws –Involve post-prison civil commitment of “high risk” sex offenders –SVP laws have been passed by legislatures in 16 states and the District of Columbia

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 8 States with SVP Civil Commitment Laws Arizona California Florida Illinois Iowa Kansas Massachusetts Minnesota Missouri New Jersey North Dakota South Carolina Texas (outpatient commitment) Virginia Washington Wisconsin District of Columbia

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 9 Why Continued Scrutiny of MnSOST-R is Important Poor test development procedures Serious implications of the test for justice and public safety in civil commitment cases

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 10 Development of MnSOST-R Small original sample (256 incarcerated sex offenders) for a 16-item test –Enrichment: included 16 recidivists who did not belong to the sample –De-selection: took out 113 familial offenders who belonged in the sample Recidivism Rate = 35% - 41% vs. usual 20% – 25% “Cut Score” > 13 used for recommendation for commitment

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 11 Validity Indicators Sensitivity at cut score of 13 =.15 1-Specificity =.02 Correlation with recidivism =.45 Area under the ROC Curve =.77 Comparative test efficiency (calculated by Wollert, 2005, using Bayes’s Theorem)

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 12 Efficiency of MnSOST vs. Other ATSRs For High Risk Offenders of Different Ages

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 13 Most Thoroughly Documented Shortcoming of the MnSOST Serious flaws have been found in all proposed experience tables –1995 SOST (Base Rate = 41%) –Replaced by MnSOST-R in 1999 Base Rate for MnSOST-R was 35% Same offenders were in both samples No explanation for the drop in base rate

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 14 Flaws in 1999 MnSOST-R Test Contents –Original Experience Table (Base Rate = 35%) –Estimated Table for samples with 21% Base Rate –Estimated Table for samples with 15% Base Rate On cross-validation the recidivism rate for those with scores >13 decreased from 88% to 44% (Wollert, 2002).

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 15 12/03: A Third Version of MnSOST Was Formulated Includes almost all of the original sample of offenders and a new sample of 220 offenders Classification was much less accurate for the new sample than it was for the original sample Who knows when the next MnSOST will appear, or why?

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 16 Another Possible Problem That Has Not Been Studied Involves the Elimination of Familial Offenders The reliability of this procedure has never been determined Raises possibility that some or all of the MnSOST validity estimates are inaccurate (Wollert, 2003)

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 17 Method for Studying This Issue Obtain data from a representative sample Re-calculate validity indicators

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 18 Bartosh et al. (2003) Reported Data for a Representative Sample N = 186 Sex offenders released from prison r with sexual recidivism =.096 (ns) ROC =.58 (ns) –This means the test does not differentiate recidivists from non- recidivists

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 19 Bartosh Shared Her Frequency Data with Waggoner and Wollert Results of Waggoner’s and Wollert’s analysis of Bartosh data –Sensitivity at cut score of 13 =.14 –1- Specificity =.067 Much larger than that for the non- representative sample (i.e.,.02) –Much less efficient than other ATSRs when used to predict recidivism for those with high scores

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 20

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 21 Discussion Validity indicators reported by MnSOST test developers are inaccurate and idiosyncratic as a result of non- representative sampling A second study on a large representative sample (N = 468) is currently underway with Barbaree and Langton to explore this problem further

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 22 Conclusion Rationale for SEPT was to identify procedures that enhance test validity –Test developers need to be aware of and follow SEPT procedures –Clinicians need to be aware of the extent to which the tests they use are based on SEPT procedures

J. Waggoner & R. Wollert2005 Western Psych. Assn. Convention April Portland, OR 23 References American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC. Bartosh, D. L., Garby, T., Lewis, D., Gray, S. (2003). Differences in the predictive validity of actuarial risk assessment in relation to sex offender type. International Journal of Offender Therapy and Comparative Criminology, 47(4), Doren, D. M., Dow, E. A. (2003). What “shrinkage” of the MnSOST-R? A response to Wollert (2002b). Journal of Threat Assessment, 2(4), Epperson, D. L., Kaul, J. D. and Huot, S. J. (1995) Predicting risk of recidivism for incarcerated sex offenders: Updated development on the sex offender screening tool (SOST). Paper presented at the 14th Annual Research and Treatment Conference of the Association for the Treatment of Sexual Abusers, New Orleans, Louisiana. Epperson, D. L., Kaul, J. D., & Hesselton, D. (1997) Final report on the development of the Minnesota Sex Offender Screening Tool-Revised (MnSOST-R). St. Paul, MN: Minnesota Department of Corrections. Epperson, D. L., Kaul, J. D., Huot, S., Goldman, R., & Alexander, W. (2003, December). Minnesota Sex Offender Screening Tool-Revised (MnSOST-R): Development, validation, and recommended risk level cut scores. St. Paul, MN: Minnesota Department of Corrections. Wollert, R. W. (2002). The importance of cross-validation in actuarial test construction: Shrinkage in the risk estimates for the Minnesota sex offender screening tool-revised. Journal of Threat Assessment, 2(1), Wollert, R. W. (2003). Additional flaws in the Minnesota Sex Offender Screening Tool-Revised: A response to Doren and Dow. Journal of Threat Assessment, 2(4), Wollert, R. W. (2005). An application of Bayes’s Theorem to age-wise sexual recidivism rates. Paper presented at the meeting of the Western Psychological Association, Portland, OR.