1. Integrity Accountability Solutions RSAT Program Performance Measures National Workshop – Chicago 2014 Jimmy Steyee Deputy Project Manager at CSR, Incorporated jsteyee@csrincorporated.com James.D.Steyee@ojp.usdoj.gov

2. 2  Performance measurement overview  Data collection process  RSAT grant program and FY2013 accomplishments Overview

3.  Satisfy Government Performance and Results Act of 1993 and the Government Performance and Results Modernization Act of 2010 (collectively referred to as GPRA) A. Accountability B. Transparency  Inform budgets  Fulfill ad hoc data requests  Draft annual and quarterly reports and GrantStat  Inform targeted TTA strategy Why report “performance” data? 3

4. 4  Performance measurement focuses on whether a program is achieving its objectives A. Performance can be defined and characterized by both quantitative (numeric) and qualitative (narrative) metrics B. “What happened?”, “What activities occurred?” C. Can be used to demonstrate activity and accomplishments  Differs from program evaluation A. Multiple types (ex. program, outcome, process) B. “Does this work?”, “How well is it working?” “Why is it working?” What is performance measurement?

5.  Measures developed collaboratively by CSR, BJA, and other stakeholders A. Process starts with the development of a logic model Identifies expected program objectives, activities, and outcomes B. Next step is developing draft measures Vetted internally and by grantees C. Final revisions made and final version released  Measures are dynamic A. Reviewed and modified periodically to ensure appropriateness BJA PM Development Process 5

6.  For Example A. Objective: Serve high risk/high need individuals B. Activity: Risk and needs assessment screening C. Outputs: # individuals screened, # individuals found to be high risk/high need D. Outcome: increase in the number and percentage of high risk/high need individuals BJA PM Development Process 6

7.  Data for about 21 programs is collected through the Performance Measurement Tool (PMT) Access at www.bjaperformancetools.org Program-specific resources are available in under “Info & Resources” including, webinars, FAQs, PMT user guide, and questionnaires  Regular submission of performance data is required as a condition of all BJA grant awards A. PMT data and reports are regularly reviewed by staff – incomplete or delinquent reporting can result in the freezing of grant funds B. Requirement is separate from all other BJA grant-related reporting (ex. GMS) Data Collection Process 7

8. PMT Completion Rate  RSAT: 97% average A. TOTAL AWARDS (not including sub-awards): 176 as of April 2014  PMT (all programs): 91% A. TOTAL AWARDS (not including sub-awards) : 3,627 RSAT Grantee Accomplishments 8

9. Top Ten States—Total Enrolled ( as of December 31, 2013 ) RSAT Grantee Accomplishments 9 #StateTotal Enrolled % of Total Enrolled 1 WV12408.15 2 TX11367.47 3 CA10837.12 4 IN9836.46 5 IL7875.17 6 MA7284.79 7 AL7104.67 8 OK6614.35 9 LA6464.25 10 GA5553.65

10. Top Ten States—Successful Individuals Completing the Program (January–December 31, 2013) RSAT Grantee Accomplishments 10 #State # Individuals Completing Program Percentage 1 TX735 6.22 2 VA715 6.06 3 IL709 6.00 4 MA665 5.63 5 CA604 5.12 6 LA597 5.06 7 IN586 4.96 8 GA542 4.59 9 MI466 3.95 10 AL456 3.86

11. 11 Residential- and Jail-Based Programs

12. RSAT Residential/Jail Program Enrollment 12

13. *High Risk/High Need Participants 13

14. Residential/Jail-Based Program Completion Rate 14

15. Residential/Jail-Based Program Completions and Exits: January–December 2013 15

16. Residential/Jail-Based Program Completions: January–December 2013 16 JailResidential Participants Who Completed the Program: Time FrameN%N% 0 to 3 Months145748123315 4 to 6 Months125842336641 7 to 9 Months2468273533 10 or More Months47287211 Total30081008206100

17. Residential/Jail-Based Program Unsuccessful Exits: January–December 2013 17 JailResidential Participants Who Did Not Complete Program: Time FrameN%N% 0 to 3 Months82885198949 4 to 6 Months12112156338 7 to 9 Months1823479 10 or More Months401684 Total9711004067100

18. Residential/Jail-Based Program Unsuccessful Exits: January–December 2013 18 JailResidential Participants Who Did Not Complete Program: ReasonsN%N% Termination for a New Charge778662 Release or Transfer to Another Facility2612785221 Death or Serious Illness263852 Voluntary Drop Out1491542010 Failure to Meet Program Requirements15216117329 Violation of Institutional Rules25827122530 Other4852466 Total9711004067100

19. Residential/Jail-Based Program Services: October– December 2013 19

20. 20 Aftercare Programs

21. Aftercare Programs: Enrollment 21

22. Aftercare Programs: Case Planning and High Risk/High Need Participants 22

23. Aftercare Programs: Program Completions and Program Exits 23

24. Aftercare Programs: Program Completions and Program Exits—January-December 2013 24

25. Aftercare Program Services: October–December 2013 25

26. Aftercare Programs: Program Completions—January- December 2013 26 Aftercare Participants Who Completed the Program: Time FrameN% 0 to 3 Months 59777 4 to 6 Months 12216 7 to 9 Months 263 10 or More Months 344 Total 779100

27. Aftercare Programs: Program Exits—January- December 2013 27 Aftercare Participants Who Did Not Complete Program: Time FrameN% 0 to 3 Months82885 4 to 6 Months12112 7 to 9 Months182 10 or More Months40 Total971100

28. Aftercare Programs: Program Unsuccessful Exits— January-December 2013 28 Aftercare Participants Who Did Not Complete Program: ReasonsN% Termination for a New Charge 13718 Release or Transfer to Another Facility 324 Death or Serious Illness 20 Voluntary Drop Out 14118 Failure to Meet Program Requirements 37448 Absconded 476 Other 466 Total 779100

29. Summary  Risk and Needs Assessment Screening and Treatment Planning  Substance Abuse Testing  Program Services  Substance abuse treatment; cognitive; behavioral, social; and vocational services  Program Length—  Residential Programs: 6-12 months  Jail-Based: at least 3 months 29

30.  If you have any questions about program performance measures or issues related to the PMT system, contact the PMT Help Desk. A. Hours: 8:30am to 5:30pm EST B. Email: bjapmt@csrincorporated.com C. Phone: 1-888-252-6867  For training and technical assistance, contact AHP.  For other grant related issues, contact your BJA grant manager. Contact Information 30

31. 31 Q&A Jimmy Steyee Deputy Project Manager at CSR, Incorporated jsteyee@csrincorporated.com James.D.Steyee@ojp.usdoj.gov jsteyee@csrincorporated.com James.D.Steyee@ojp.usdoj.gov