1. 1 © 2009 Council of Juvenile Correctional Administrators Performance-based Standards (PbS) for Youth Correction and Detention Facilities Using Performance-based Standards (PbS) to Measure Evidence-based Practices Correctional Education Association March 31, 3009 Annapolis, MD

2. 2 © 2009 Council of Juvenile Correctional Administrators 2 PbS: Measuring Performance and Improvement Ned Loughran, executive director/project director CJCA National non-profit organization dedicated to improvement of youth correctional services and practices so youths succeed when they return to the community Unites nation’s youth correctional CEOs to promote best practices, address common concerns and provide leadership for juvenile justice Directs several grant projects: MacArthur Foundation Model for Change, CJCA Yearbook OJJDP Mental Health Model, New Directors Seminar Committed to expanding the adoption of PbS as a best practice to improve conditions of confinement

3. 3 © 2009 Council of Juvenile Correctional Administrators Presentation Overview How PbS was developed The scope of PbS: Standards and Outcomes How PbS works PbS Tools for the Field Outcome Measure Examples NDTAC Recommendations for PbS/CJCA Surveys Questions and Answers www.pbstandards.org for more information www.pbstandards.org

4. 4 © 2009 Council of Juvenile Correctional Administrators Performance-based Standards (PbS) Information, outcome measures, reports: What’s going on Injuries, climate, fairness, staff-youth relationships Risk management Suicidal behavior, screening, isolation What works, what doesn’t Education, programming, health/mental health services Evaluation and evidence-based practices Treatment, behavior management

5. 5 © 2009 Council of Juvenile Correctional Administrators Performance-based Standards (PbS) Conditions of Confinement Quality of life for youths and staff Making sure facilities are safe, hold youths accountable and provide rehabilitation that prevents future crime

6. 6 © 2009 Council of Juvenile Correctional Administrators 1994 Conditions of Confinement Study About 1,000 secure facilities and found “substantial and widespread deficiencies:” High rates of youths and staff getting hurt High rates of suicidal behavior Few timely or professional health screenings High levels of staff turnover Adherence to existing standards did not mean “better” facility

7. 7 © 2009 Council of Juvenile Correctional Administrators Participants: October 2008 178 facilities 25 states, 9 counties 116 correction 48 detention 14 assessment centers 23 all-female facilities 73 facilities < 50 youths 26 facilities 50-99 youths 79 facilities > 100 youths 14,000 youths

8. 8 © 2009 Council of Juvenile Correctional Administrators How PbS Works: The Blueprint A set of seven goals and 27 standards for corrections and 19 standards for detention facilities to assess: Safety Order Security Health and mental health Programming Justice Reintegration (Corrections Only) Performance toward meeting each standard is measured using one or more outcome measures, which are compared over time and to other participating facilities. PbS is a standards based performance improvement measurement system

9. 9 © 2009 Council of Juvenile Correctional Administrators Each of the seven areas are formatted in the following manner: Area: XXX Goal XXX1Performance Standard PbS Standard Layout Outcome MeasuresExpected PracticesProcesses Rates, frequencies, numbers that show change in status, occurrence or prevalence. Practices expected to produce higher quality outcomes based on available research, consensus of professional opinion and past PbS experience. Policies, forms, tests used to describe or perform practices.

10. 10 © 2009 Council of Juvenile Correctional Administrators PbS Goals Safety: To engage in management practices that promote the safety and well-being of staff and youths. Order: To establish clear expectations of behavior and an accompanying system of accountability for youths and staff that promote mutual respect, self discipline and order. Security: To protect public safety and to provide a safe environment for youths and staff, an essential condition for learning and treatment to be effective. Programming: To provide meaningful opportunities and services to youths to improve their educational and vocational competence, to effectively address underlying behavioral problems and to prepare them for responsible lives in the community.

11. 11 © 2009 Council of Juvenile Correctional Administrators PbS Goals Justice: To operate the facility in a manner consistent with principles of fairness and that provides the means of ensuring and protecting youth’s and family’s legal rights. Health and Mental Health: To identify and effectively respond to residents’ health, mental health, and related behavioral problems throughout the course of confinement through the use of professionally appropriate diagnostic, treatment, and prevention protocols Reintegration: To prepare youths for successful reintegration into the community while they reside at the facility

12. 12 © 2009 Council of Juvenile Correctional Administrators PbS Outcome Measures AreaCorrectionsDetention Safety 14 Order 12 Security 77 Health 147 Programming 2712 Justice 77 Reintegration 240 Totals27 Standards/ 105 OMs 19 Standards/ 59 OMs

13. 13 © 2009 Council of Juvenile Correctional Administrators April and October are data collection months. Draft Site report is received within 48 hours of close of data entry. Data corrections are made to anomalies, not recorded and outliers Final Site Reports are generated along with summary reports for all data collection forms 14 days after receipt of the draft reports Improvement plan is developed and entered into website with targeted outcome measures. How PbS Works: Improvement Cycle

14. 14 © 2009 Council of Juvenile Correctional Administrators Data Collection Administrative Form 1 per site, 46 questions Incident Reports All incident reports for data collection period (12,849) Youth Record 30 random YR, 93 questions (5274) Youth Climate Survey 30 random youths, 38 questions (5668) Staff Climate Survey 30 random staff, 38 questions (5999) Youth Exit Interview All youths released since last data collection, 24 questions (5899) Ongoing data entry

15. 15 © 2009 Council of Juvenile Correctional Administrators Site Reports and Analysis Divided Between Areas: Safety, Order, Security, Health/ Mental Health, Programming, Reintegration Corrections: 105 outcomes Detention: 59 outcomes Compare over time, to the Field Average

16. 16 © 2009 Council of Juvenile Correctional Administrators Components of a Sites FIP Targeted Outcome Measures Targeted Outcome Measure Goals What is the problem? Action Steps Progress Notes Ongoing Review by Facility Administrator, PbS State Coordinator, and PbS Coach Facility Improvement Plans

17. 17 © 2009 Council of Juvenile Correctional Administrators Measuring Performance PbS outcome measures adhere to definition of performance outcomes as rates, frequencies, numbers that show change in status, occurrence or prevalence PbS outcomes are measured twice a year to reflect change from one data collection period to the next PbS reports performance to users in many ways: performance over time, in comparison to the field average of other participating facilities, performance on outcomes targeted for improvement and outcomes critical to safe and effective operations

18. 18 © 2009 Council of Juvenile Correctional Administrators Facility Site Reports and Performance Profiles Provide a compass for future improvement opportunities Create a culture where it is accepted that data was not going to be used to “catch” people Reliable, accurate data will be used to manage and drive improvement

19. 19 © 2009 Council of Juvenile Correctional Administrators PbS Coaches use Performance Profiles and Site Reports to guide sites to opportunities for improvement Sites develop Facility Improvement Plans after each data collection Ask sites to develop PbS sampling, definition, comprehensive reporting, and thorough data collection competencies first. Facility Improvement Plans PbS Sustained Improvement

20. 20 © 2009 Council of Juvenile Correctional Administrators Components of a Sites FIP FIP Status: Open or Closed Targeted Outcome Measures Targeted Outcome Measure Goals What is the problem? Action Steps Progress Notes Ongoing Review by Facility Administrator and PbS State Coordinator Facility Improvement Plans

21. 21 © 2009 Council of Juvenile Correctional Administrators PbS Technical Assistance Web site, data collection and data quality Technical assistance with data collection, site reports and facility improvement plans, both by phone and email Linkage to OJJDP Training and Technical Assistance Distance Learning Calls Consultants/coaches, staff Assist with Site Report Analysis, FIP development, coaching Data quality review site visits

22. 22 © 2009 Council of Juvenile Correctional Administrators Why PbS Works: PbS gives facilities data, twice year, showing levels of safety, order, security, programming, health/mental health services, justice and reintegration PbS improvement process guides facilities in ways to use the information to identify what works and celebrate success as well as see what is not working and a structured path to create change Improves accountability: facilities have data to share Prevents future incidents, lawsuits by improving compliance with best practices and high standards Identifies progress over time, compared to other facilities Ongoing, personalized customer support Access to resources, networking across USA (Yale School of Management’s Survey of PbS Users)

23. 23 © 2009 Council of Juvenile Correctional Administrators How do we know? Field testimonials: word-of-mouth and money 2004 Innovations in American Government Award CRIPA analysis: Legal protection Mapped 10 years of investigations with PbS Conclusion: Facilities that implement PbS as intended and should not expect a CRIPA investigation because CRIPA investigates based on failure to meet the bare minimum Constitutional standards and PbS sets the highest standards for operational success. PbS Research: Each kid’s individual experience best way to impact facility safety

24. 24 © 2009 Council of Juvenile Correctional Administrators Field Responses: Directors, managers and staff are using data to understand facility performance and services PbS demonstrates data can be provided within the context of improvement: PbS site reports identify the good and not-so- good; Expected practices and processes provide a roadmap of ways to improve FIP process integrates continuous work and monitoring over time to demonstrate accountability and effectiveness

25. 25 © 2009 Council of Juvenile Correctional Administrators Benefits of PbS Provide sites and agencies with a system of continual improvement Tool to measure and track key indicators of facility performance and services Able to compare performance to similar participating facilities across the country Allow facilities to demonstrate success with data rather than anecdotes Become more accountable and gain public support

26. 26 © 2009 Council of Juvenile Correctional Administrators Outcome Measure Example: Order Goal: To establish clear expectations of behavior and an accompanying system of accountability for youths and staff that promote mutual respect, self discipline and order. Standards: Maximize responsible behavior by youth and staff and conformance to facility rules; Minimize the facility’s use of restrictive and coercive means of responding to disorder; Maximize opportunities for youths to participate in activities and programs. Outcome Measures: Incidents of youth misconduct; use of physical restraint; use of mechanical restraint; use of isolation or room confinement and; average duration of isolation or room confinement.

27. 27 © 2009 Council of Juvenile Correctional Administrators Order 3: Physical Restraint Use The denominator is constructed using the total number of youth population for the month (2533) The numerator is constructed from incident reports of all physical restraints Raw score for the October 2008 data collection was 23 physical restraints

28. 28 © 2009 Council of Juvenile Correctional Administrators Outcome Measure Example: Programming Goal: To provide meaningful opportunities and services to youths to improve their educational and vocational competence, to effectively address underlying behavioral problems and to prepare them for responsible lives in the community. Standards: Provide an education program that is tailored to each youth’s education level, abilities, problems, and special needs, and improves education performance and vocational skills while confined. Address the behavioral problems of confined youths by developing and implementing a level system and associated programming that prepares youths for progressively increased responsibility and freedom in the community and that promotes healthy life choices. To provide a support system to ensure that services are gender ‐ specific, culturally sensitive, language appropriate and tailored to fit the individual needs to the youths.

29. 29 © 2009 Council of Juvenile Correctional Administrators Programming 2: Reading Scores Percent of youths whose reading scores increased between admission and discharge

30. 30 © 2009 Council of Juvenile Correctional Administrators Programming 6: Vocational Programming Percent of youths whose records indicate that they received the vocational skills programming prescribed by their individual treatment plans

31. 31 © 2009 Council of Juvenile Correctional Administrators NDTAC Recommendations for PbS/CJCA Surveys Include education as a heading/category of its own, separated from the programming goal Creating three education standards 1.General education 2.Special education 3.Vocational education Include additional items that focus on:  Special education services  Advanced student outcomes  Educational transition  Teacher qualifications

32. 32 © 2009 Council of Juvenile Correctional Administrators Each of the seven areas are formatted in the following manner: Area: XXX Goal XXX1Performance Standard PbS Standard Layout Outcome MeasuresExpected PracticesProcesses Rates, frequencies, numbers that show change in status, occurrence or prevalence. Practices expected to produce higher quality outcomes based on available research, consensus of professional opinion and past PbS experience. Policies, forms, tests used to describe or perform practices.

33. 33 © 2009 Council of Juvenile Correctional Administrators Standard 1: General Education Outcome MeasuresExpected PracticesProcesses Percent received a math/reading test at admission. Percent received the educational programming prescribed by their individual treatment plans. Percent of non-English speaking youth who have treatment plans written in the appropriate language. Percent whose individual treatment plans have monthly progress notes. Receive math/reading tests to determine grade level within the 1 st day Trained and qualified staff applies the math/reading tests and interpret findings The education program is provided 12 mos a year and for the number of hours per day specified by state law. The education records contain info from their most recent school, arrived within 14 days The facility uses aggregate and summary education data to develop a plan to improve education programming. Youths held in isolation receive education programming and materials. Written individual treatment plans are based on the results of reading and math tests and the education, social skills and vocational skills assessments. Written math/reading tests that help determine grade-level exist Math/reading tests are age- appropriate and externally normed and validated. Facility policy dictates that written individual treatment plans address education, social skills and vocational skills. Case-level data on youths’ education status, needs, and performance are aggregated at least annually. Facility policy provides for the education for youths held in isolation. The provision of educational materials for youths held in isolation is stipulated in facility policy. A record of materials provided to youths held in isolation exists. Educational Programming: Currently in PbS

34. 34 © 2009 Council of Juvenile Correctional Administrators Standard 1: General Education Outcome MeasuresExpected PracticesProcesses Length of average school day (in hours) Average class size for: o General Curriculum o Special Education o Vocational Percent of high school- level courses from which credits are transferrable to community high schools. Number of hours per week the school library is available to students. The school day is 6 hours exclusive of lunch and count related activities. Age-eligible students are able to earn high-school course credits that can be transferred to local community high schools. There is an active school library staffed by a full or part-time librarian. Male and female residents have access to the same educational opportunities. Educational Programming: Recommendations

35. 35 © 2009 Council of Juvenile Correctional Administrators Standard 1: General Education Outcome MeasuresExpected PracticesProcesses Percent of youths confined for over 6 months whose math/reading scores increased between admission and discharge. Youth are re- assessed in math and reading every six months and at discharge. Facility policy dictates that youths are re- assessed in math and reading. Student Academic Outcomes: Currently in PbS

36. 36 © 2009 Council of Juvenile Correctional Administrators Standard 1: General Education Outcome MeasuresExpected PracticesProcesses Percent of age-eligible students earning a high school diploma in the most recently completed school year. Percent of age-eligible students earning a GED in the most recently completed school year. Youth are tested in math and reading every 3 months. Student Academic Outcomes: Recommendations

37. 37 © 2009 Council of Juvenile Correctional Administrators Standard 1: General Education Outcome MeasuresExpected PracticesProcesses Rate of percent of minority staff to percent of minority youth. Teachers/Staff: Currently in PbS

38. 38 © 2009 Council of Juvenile Correctional Administrators Standard 1: General Education Outcome MeasuresExpected PracticesProcesses Percent of teachers that are certified in the area in which they are teaching. Percent of teachers with any certifications. Percent of ALL school staff that have college degrees. Students are taught by State Department of Education (SDE) credentialed teaching staff or students are taught by teaching staff that meet the highly qualified teaching standards set up by NCLB A guidance counselor is a part of the educational staff. A transition coordinator is a member of the educational staff. Teachers/Staff: Recommendations

39. 39 © 2009 Council of Juvenile Correctional Administrators Standard 1: General Education Outcome MeasuresExpected PracticesProcesses Percent of youth who have an institutional education file in the school office. Percent of youth enrolled in an educational program that issues a report card. Percent of youth admitted whose educational records are requested within 3 days of admission. Average number of days between a youth’s arrival and when the school record arrives from community school district. Average number of days between a youth’s discharge and when the facility school records were released (mail/fax/hand carried) to his community school or district. Percent of individual facility re-entry plans (individual treatment plans) that reflect educational re-entry activities/priorities/goals. Educational records are maintained on youth while enrolled in facility school, including the issuing of report cards every grading period. The facility school establishes formal communication with the community school youth last attended. Records are transferred from the facility to the new school within 7 days of discharge. Future educational plans are considered by facility case management staff as demonstrated by youth’s re-entry plans. Transition/Reintegration: Recommendations

40. 40 © 2009 Council of Juvenile Correctional Administrators Standard 2: Special Education Outcome MeasuresExpected PracticesProcesses Education and program staff sign off on the written individual treatment plans and/or the individual education plan. Copies of IEPs are distributed to the staff responsible for implementing them. Facility policy dictates that individual education plans (IEPs) are developed for appropriate youth as mandated by federal education regulation. Currently in PbS

41. 41 © 2009 Council of Juvenile Correctional Administrators Standard 2: Special Education Outcome MeasuresExpected PracticesProcesses Percent of students arriving with IEPs developed within last 12 months. Percent of students needing new or updated IEPs based on intake assessment. Percent of IEPs completely implemented within 10 calendar days of the IEP meeting. Percent of IEP meetings held. Percent of parents sent IEP notification letters of IEP meetings. Percent of IEP meetings in which parents attended (in-person or by phone). Percent of educational transitional services identified on the youth’s IEP that are coordinated with the youth’s institutional treatment plan. Existing student IEPs are reviewed for their currency and adequacy. There exists an active child search/child find staff/committee to determine which youth are educationally disabled upon entry to the facility. The facility has access to the state- wide special education database/network. An IEP meeting is held within 30 days of the youth’s enrollment in facility school. Parents are invited to their child’s IEP meeting Parents provide input to their child’s IEP. Recommendations

42. 42 © 2009 Council of Juvenile Correctional Administrators Standard 3: Vocational Education Outcome MeasuresExpected PracticesProcesses Percent of youths who received a vocational assessment by qualified staff. Percent received the vocational skills programming prescribed by their individual treatment plans. Percent completed a vocational skills curriculum. Youths receive a vocational assessment within 30 days after admission Trained and qualified staff apply the vocational assessment and interpret the findings Vocational skills classes are held as scheduled. A written vocational assessment for youths exists The written vocational assessment is age- appropriate and externally normed and validated. An age-appropriate vocational skills curriculum exists. Currently in PbS

43. 43 © 2009 Council of Juvenile Correctional Administrators Standard 3: Vocational Education Outcome MeasuresExpected PracticesProcesses Percent of vocational courses that follow an established vocational education curriculum that is recognized by the state education agency. Percent of youth enrolled in recommended vocational programming as determined/recommended by his vocational assessment. Percent of vocational coursework hours that are transferrable to a career college or secondary vocational program of instruction. Percent of youth in vocational programs that earned transferrable credit hours. Students are afforded vocational programming comparable to vocational programs operated in the community or community school. Students receive appropriate vocational instruction that is aligned with interests and/or aptitude. A written policy/procedure for admission to vocational programs exists. A written policy on the type of vocational offered: a secondary school model as offered in community schools, or a vocational/technical school model as offered in community/technical colleges. Recommendations

44. 44 © 2009 Council of Juvenile Correctional Administrators Recommendations for PbS/CJCA Youth Surveys Programs/Education I am in class (6 hours or more), five days per week. I have what I need to complete my lessons in my classes. I have received a copy of my most recent report card. The school library is available for me to use during school hours.

45. 45 © 2009 Council of Juvenile Correctional Administrators Recommendations for PbS/CJCA Youth Surveys Classes and Teachers Overall, the courses here are challenging. Overall, the teachers here care about my education. Overall, the teachers here help me if I don’t understand something in my lesson.

46. 46 © 2009 Council of Juvenile Correctional Administrators Recommendations for PbS/CJCA Youth Surveys Transition I have talked with a school staff member about my educational plans when I am released. School staff encourage me to get a high school diploma. School staff encourage me to get a GED.

47. 47 © 2009 Council of Juvenile Correctional Administrators PbS Website and Email Address www.pbstandards.org help@pbstandards.org Availability of Trial Facility to experience PbS website design, data collections, outcome measures and reports.