1. RESTORATIVE JUSTICE EVALUATIONS: WHAT WORKS?
Introduce yourself and your department/school
Kyle Meditz
School of Criminology and Criminal Justice
Arizona State University

2. Project Overview
Use existing evaluative research to answer the question, “what works?” about restorative justice programs.
Compare program outcomes and program effectiveness in the context of methodological rigor.
Inform future evaluative research in terms of design and conceptualization of success.

3. Understanding what works in Restorative Justice Programs is important because:
Restorative Justice use is becoming more prevalent in and outside the Criminal Justice System.
Critical to determine current state of evaluations to assess their true impact on offenders, victims, and communities.

4. Current Study Foci: What DVs are used in RJ evaluations and why
2) Effectiveness of evaluations using authors’ operational definitions of success and significance
3) *Methodological strength/rigor of RJ evaluations
4) Compare program success against methodology strength between specific groups

5. Method: Systematic Review of Restorative Justice Evaluations
Data Collection Steps:
Key terms were used to search electronic databases
Secondary search using article bibliographies
Articles were screened out based on abstract content
Total sample size of 45 studies.

6. Method: Quantitative Data Collection and Coding Procedure
Quantitative data from articles were gathered using an original coding sheet which included sections on:
Program Characteristics (program type, participants allowed)
Sample/Offender Characteristics (age, race, gender, criminal history, offense type)
Research Design (cross-section, longitudinal, quantitative, qualitative, mixed-methods)
DVs (recidivism, offender/victim satisfaction, restitution compliance, etc.)

7. Method: Qualitative Data Collection and Coding Procedure
Thematic coding of qualitative evaluation outcomes
Certain evaluations were completely qualitative in nature.
Evaluator rationale for dependent variable(s) used
Use of dependent variables should be reflective of assessing program goals. By coding this information context is provided to explain the variation and discrepancies within outcome variables.

8. Method: Measuring Methodological rigor using the Maryland Scientific Methods Scale
SMS Scoring Table
(Sherman et al., 2002)

9. Results: Composition of DVs

10. Results: Composition of DVs Continued
Rationale for DV (n=20):
55% of responses suggested that recidivism was used because it was easy/quantifiable
45% victim perceptions (i.e., perceived impact on offender, satisfaction, and fear) important aspects of victim recovery

11. Results: Success Outcomes
53.3% (N=24) TX worked compared to control in part or in full
Majority of these evaluations were conducted on Victim-Offender-Mediation/Reconciliation
Majority measured recidivism and victim satisfaction and/or fear as outcome variables.

12. Results: Methodological Rigor as Measured by SMS
57.7% (N=26) received an SMS score in the 3-5 range.
Considered moderate to excellent ranking.
However, when successful outcomes and methodological strength were compared…
Only 35.6% (N=16) received both an SMS score in the 3-5 range and had significant findings for outcome variable(s) compared to control groups.

13. Chi-square Tests Results Series 1:
Test for significant differences between SMS scores of different groups.
Offender participation: voluntary and non-voluntary
Country: US and non-US studies

14. Chi Square Results: Offender Participation, country of origin, and SMS Scores

15. Chi-square Tests Results Series 2:
Test for significant differences between studies with significant vs. non-significant outcomes based on offender:
age group
offense type
criminal history

16. Offender Type, Offense Type, Criminal History and Successful outcomes

17. Summary of main findings:
Lack of standardization in dependent variable measurement
Victim-offender mediation was found to have the most programmatic success
Offender non-voluntary participation was not predictive of poor outcomes
Nor were prior criminal history, offender age group or offense type

18. Implications Despite small sample size…
Overall findings were supportive of positive restorative justice impact
However the number of successful program evaluations was reduced when moderate to excellent methodological strength was considered.
Lack of statistically significant differences between offender characteristics support increased use with adult, serious, and non-voluntary offender participation.

19. Future Research
Calculate effect sizes to compare program success and method strength.
Focus more attention on victim-oriented dependent outcomes.
How do we capture restoration of harm for all stakeholders?