Researching Sex Offenders: A Workshop on Conceptualizing and Implementing Sex Offender Research Projects R. Karl Hanson Public Safety Canada Presentation at the 13 th Annual Conference of the NYATSA, Saratoga Springs, NY, May 14, 2008

Big Questions Impact of criminal justice and social policy interventions (e.g., community notification, residency restrictions, civil commitment, risk assessment) Assessment of reduced risk in high risk offenders Active components in sexual offender treatment Social policy for prevention on onset

Evaluation of Sex Offender Treatment Programs

Collaborative Outcome Data Committee Formed in 1997 Goals: –Define standards for research on treatment outcome for sexual offenders (develop consensus) –Organize existing sexual offender outcome studies –Promote high quality evaluations

CODC contributors Anthony Beech Darren Bisshop Guy Bourgon Dawn Fisher R. Karl Hanson Andrew Harris Calvin Langton Roxanne Lieb Janice Marques Michael Miner William Murphy Michael Seto Vernon Quinsey David Thornton Pamela Yates

CODC Study Quality Guidelines Structured rating scale Definition of study quality –“…judgement of minimal bias can be made with high confidence.” 20 items (plus 1 additional item rated for cross- institutional designs) –Items fall under 7 categories –Items assess either confidence OR bias (including direction of bias)

I) Administrative control of independent variables Defining treatment (confidence) Defining comparison group (confidence) Miscellaneous incidental factors (bias)

II) Experimenter expectancies Experimenter involvement (bias) Blinding in data management (bias)

III) Sample size Sample size of treatment (confidence) Sample size of comparison (confidence) Sample size of institutions (confidence) (for cross-institutional designs only)

IV) Attrition Subject selection (bias) Program attrition (bias) Intent-to-treat (bias) Attrition in follow-up (bias)

V) Equivalency of groups A priori equivalency of groups (bias) Adequacy of search of differences (confidence) Findings on group differences (bias)

VI) Outcome variables Length of follow-up (confidence) Validity/reliability of recidivism information (confidence) Equivalency of follow-up (bias)

VII) Correct comparison conducted Data dredging (confidence) Effectiveness of statistical controls (confidence) Computation of least bias comparison (bias)

Global Rating All items considered in making overall judgment of bias and confidence –Same three-point scales as individual items Bias and confidence are considered separately

Global Rating Categories Strong –High confidence AND negligible bias Good –High confidence and some bias, OR –Some confidence and negligible bias Weak –Some confidence and some bias Reject –Little confidence, OR –Considerable bias

Reliability Study 1: Students 2 senior undergrad students Approximately one week training (8 practice studies) 10 real studies rated independently

Results: Global Ratings Overall 9/10 (ICC = 0.95) Global confidence10/10 (ICC = 1.00) Global bias 9/10 (ICC = 0.69) Direction of bias 7/10

Reliability Study 2: Experts 12 Experts in sex offender research evaluation No training on guidelines 10 hypothetical studies ranging in quality Rated 1-6 studies each (3 ratings per study)

Expert Raters Guy Bourgon Andrew Harris Grant Harris Niklas Langstrom Roxanne Lieb Ruth Mann Robert McGrath William Murphy Vernon Quinsey Marnie Rice David Thornton Pamela Yates

Expert Reliability –Some agreement on individual items –No agreement on global ratings

Collaborative Data Outcome Committee. (2007). Sex offender treatment outcome research: Guidelines for Evaluation (CODC Guidelines). Part 1: Introduction and overview. Corrections User Report No Ottawa: Public Safety Canada. Collaborative Data Outcome Committee. (2007). The Collaborative Outcome Data Committee’s Guidelines for the evaluation of sexual offender treatment outcome research. Part 2: CODC Guidelines. Corrections User Report No Ottawa: Public Safety Canada.