1. Using Perkins Data To Improve CTE Sharon Enright, Ph.D. Ohio Department of Education Office of Career-Technical Education 1 NCLA Conference October 1, 2010

2. Greetings from Ohio The Buckeye State! 2

3. Four areas of CTE Accountability: Secondary (ODE) Postsecondary (OBR) Adult (ODE and OBR) Tech Prep (ODE and OBR) 3 Ohio’s CTE Accountability System

4. Negotiated State Targets with OVAE Obtained input from Locals prior to negotiating State targets. FY08 – Secondary only (1S1, 1S2, 4S1) FY09 – Secondary, Postsecondary, Adult FY10 and FY11 – Secondary, Postsecondary, Adult 4

5. Renegotiated State Targets with OVAE FY09 – Renegotiated some Postsecondary and Adult targets FY10 and FY11 – Renegotiated Placement targets (5S1, 4P1, 4A1) –State and county unemployment rates continue to go up 5

6. Statewide Longitudinal Data System (SLDS) Perkins data is submitted by Ohio locals in SLDS Secondary – Education Information Management System (EMIS), PK-12 Postsecondary – Higher Education Information Data System (HEI), 13-20+ Ohio is working on an integration plan for PK-20+ SLDS 6

7. Local and State Perkins Data LOCALS – Submit course, student-level and other data to State via SLDS. STATE – Aggregates LOCAL data: –Calculates LOCAL performance results – Reports some local Secondary results to USDOE via EDFacts. Publishes local performance reports. –Calculates STATE performance results – Reports state results to USDOE/OVAE via CAR & EDFacts. Publishes state performance report..7

8. FY2009 State Performance No State Improvement Plans Needed Secondary – Exceeded State targets for all indicators Postsecondary – Exceeded State targets for all indicators Adult –Did not meet State target for Placement (4A1), but exceeded 90% 8

9. FY2009 Statewide Performance Reports Goal – Create user-friendly State reports Publish on the Web –Ohio Department of Education (ODE) –Ohio Board of Regents (OBR) Review reports with State and local CTE staff 9

10. Ohio is on a journey to improve CTE student performance… And we’re not on this journey alone. 10

11. Ohio’s strategies to improve Statewide performance results should also help improve Local performance results. 11

12. 12 CTE Performance, Data and Accountability Web site

13. 13 Secondary CTE Performance Reports

14. Review of Secondary State Data 14

15. State Staff Review – Meeting #1 Reviewed FY09 Enrollment Data State Performance Report Also pulled additional data to review 15

16. 16 Career-Technical Education (CTE) and Tech Prep Student Counts (unduplicated) In each of the following categories, unduplicated means a student is counted only one time in a category, but a student may be counted in more than one category. A student may be counted only as a CTE Participant; or as a CTE Participant and a CTE Concentrator; or as a CTE Participant, a CTE Concentrator and a CTE Concentrator Who Left Secondary Education. 2007- 2008 2008- 2009 Total CTE Participants* A secondary student who has earned credit in one (1) or more courses in any career and technical education (CTE) workforce development program area. 137,961127,085 Total CTE Concentrators A secondary student who has completed a minimum of 50% of the high school credits allowed for a single career and technical education (CTE) workforce development program (e.g., health sciences or marketing), and has enrolled for additional credit at the secondary level. 32,27232,747 Total CTE Concentrators Who Left Secondary Education CTE Concentrators who have graduated or withdrawn from high school. 26,23027,098 Total Secondary Tech Prep Participants A secondary student who has earned credit in one (1) or more courses in any career and technical education (CTE) Tech Prep workforce development program area. Not Calculated 22,989 Total Secondary Tech Prep Concentrators A secondary student who has completed a minimum of 50% of the high school credits allowed for a single career and technical education (CTE) Tech Prep program (e.g., health sciences or marketing), and has enrolled for additional credit at the secondary level. 8,9089,870 Total Secondary Tech Prep Concentrators Who Left Secondary Education Secondary Tech Prep Concentrators who have graduated or withdrawn from high school. 7,7108,530 * CTE Participant is calculated differently in 2008-2009 than in previous years. FY09 CTE Enrollment Data – Pg. 2

17. 17 2008-2009 Statewide Student Subgroup Counts (unduplicated) Statewide student counts are disaggregated by student subgroups. These data support statewide accountability and continuous improvement initiatives. Student Counts by Student Subgroups CTE Participants CTE Concentrator s CTE Concentrator s Who Left Secondary Education Secondary Tech Prep Participants Secondary Tech Prep Students (Concen.) Secondary Tech Prep Students (Concen.) Who Left Secondary Education All Students127,08532,74727,09822,9899,8708,530 Male72,57117,81014,51212,9715,4584,669 Female54,51414,93712,58610,0184,4123,861 American Indian or Alaskan Native1895643291918 Asian or Pacific Islander99625820122110588 Black (not Hispanic)19,5744,1373,0482,6171,092880 Hispanic2,991649458439189154 White100,74127,15822,96619,3038,3147,269 Unknown – Multi-Racial2,594489382380151121 Individuals with Disabilities22,6107,0855,4182,9581,2461,044 Economically Disadvantaged43,82210,2168,0106,5472,6092,129 Limited English Proficient1,224180981095034 Single Parents1,212615523266129109 Migrant Status2673211 CTE Nontraditional Enrollees23,9506,7285,5415,0082,0711,805 FY09 CTE Enrollment Data, Disaggregated – Pg. 3

18. State Staff Review – Meeting #1 Reviewed FY09 Enrollment Data CTE Participant Count – Lower in FY09 than FY08 (New calculation method in FY09) CTE Concentrators – Suspect under-reporting CTE Student Sub-groups: –Students with Disabilities – CTE has higher enrollment rate than general school population, grades 9-12 –Single Parents – Need to improve identification and reporting 18

19. State Staff Review – Meeting #2 Did Ohio meet all state targets? Did Ohio meet 90% of state targets? How did student sub-groups perform? How did the LOCALS perform? 19 Reviewed FY09 Performance Data

20. 20 FY09 Performance Data– Pg. 2 Did We Meet All Targets? Did We Meet 90% of All Targets? Secondary CTE Workforce Development Perkins Core Indicators of Performance Definitions for the indicators of performance are in the Ohio Perkins Five-Year State Plan for Career-Technical Education. The Ohio Department of Education, Office of Career-Technical Education, reached agreement with the U. S. Department of Education on state performance targets for each Perkins core indicator of performance. Perkins Core Indicators of Performance 2007-2008 State Performance Rate 2008-2009 State Performance Rate 2008-2009 State Performance Target Met 90% of 2008-2009 State Performance Target Academic Attainment—Reading/Language Arts (1S1) 93.89%93.31%87.00%Yes Academic Attainment—Mathematics 2 (1S2) 92.09%91.68%83.00%Yes Technical Skill Attainment, (2S1) n/a63.63%61.00%Yes Secondary School Completion 3,4 (3S1) n/a95.40%93.00%Yes Regular or Honors Diploma n/a95.23% General Educational Development (GED) Diploma 5 n/a0.17% Student Graduation Rates 4 (4S1) 93.38%94.63%73.60%Yes Secondary Placement 3,4 (5S1) n/a88.49%87.00%Yes Postsecondary Education or Advanced Training n/a53.00% Employment 6 n/a58.22% Military 6 n/a3.93% Apprenticeship 6 n/a3.25% Placement Status Known 7 n/a90.07% Nontraditional Participation 2,3 (6S1) n/a26.67%20.00%Yes Nontraditional Completion 2,3 (6S2) n/a22.81%17.00%Yes [1] [1] If 90% of the state performance target is not met for any Perkins core indicator of performance, Ohio must submit an improvement plan to the U.S. Department of Education. [2] [2] 2008-2009 results are based on CTE Concentrators who left secondary education in 2008-2009. [3] [3] Performance results were not calculated for these indicators and there were no performance targets in 2007-2008. [4] [4] 2008-2009 performance results are based on CTE Concentrators who left secondary education in 2007-2008 (including summer graduates). [5] [5] Secondary School Completion disaggregated indicator data – a student is included in only one category of disaggregated data. [6] [6] Secondary Placement disaggregated indicator data – a student may be included in more than one category of disaggregated data. 7 Percent of CTE Concentrators who left secondary education in 2007-2008 whose placement status was known. Not an indicator of performance.

21. 21 FY09 CTE Performance Data, Disaggregated – Pg. 3 2008-2009 Statewide CTE Performance Rates by Student Subgroup Statewide performance rates are disaggregated by student subgroups on each Perkins core indicator of performance. These data support statewide accountability and continuous improvement initiatives. (1S1) (1S2) (2S1) (3S1) (4S1) (5S1) (6S1) (6S2) All Students93.31%91.68%63.63%95.40%94.63%88.49%26.67%22.81% Male91.66%90.89%60.83%95.26%94.43%89.22%5.05%5.41% Female95.22%92.60%67.96%95.57%94.86%87.63%55.89%43.11% American Indian or Alaskan Native93.02%79.07%58.33%97.37%94.59%82.86%30.40%32.50% Asian or Pacific Islander96.02%95.52%72.50%95.41%93.26%88.70%29.42%27.78% Black (not Hispanic)90.14%85.84%51.53%92.59%93.00%76.72%28.34%24.42% Hispanic93.44% 46.40%93.10%92.42%81.96%28.99%25.97% White93.69%92.39%65.43%95.86%94.91%90.34%26.35%22.49% Unknown – Multi-Racial94.49%92.91%62.75%95.47%94.41%82.27%27.45%23.85% Individuals with Disabilities68.18%61.81%47.63%95.00%93.88%79.90%20.80%18.95% Economically Disadvantaged88.42%85.52%56.54%93.37%92.67%81.76%27.13%23.35% Limited English Proficient93.75%92.71%32.50%84.30%84.26%82.83%26.76%18.29% Single Parents*87.95%86.23%27.07%23.96% Migrant Status100.00% 50.00%66.67% 100.00%18.75%33.33% CTE Nontraditional Enrollees94.49%92.05%58.80%95.96%95.64%87.56% Secondary Tech Prep Concentrators97.51%96.34%64.00%97.01%96.18%92.54%22.80%22.77% * Single Parents data collected for the first time in FY2009. Not collected for indicators using one-year lag data (2S1, 3S1, 4S1 and 5S1).

22. 22 CTE Indicators of Performance # Locals Did Not Meet 90% Academic Attainment—Reading (1S1)1 Academic Attainment—Mathematics (1S2)1 Technical Skill Attainment (2S1)23 Secondary School Completion (3S1)2 Student Graduation Rates (4S1)0 Placement (5S1)5 Nontraditional Participation (6S1)7 Nontraditional Completion (6S2)8 FY09 Local Performance – 35 Did Not Meet 90% of One (1) or More Targets

23. What can State Staff do to help improve 2S1? 23 Improve guidance and technical assistance provided. Improve published resources – Example is CTE Assessment Matrix: –State CTE Assessments –Industry Assessments Improve local data verification reports.

24. Local Administrators Conference 24 Did Ohio meet all state targets? Did Ohio meet 90% of state targets? How did student sub-groups perform? How did the LOCALS perform? Reviewed FY09 Performance Data, same questions

25. Local Administrators Discussion 25 Explain more about CTE Participant. Explain more about CTE Concentrator. How is 3S1 different than 4S1? Explain more about 5S1. How is 2S1 calculated? Which students have to take technical assessments? When do we report what data? MUCH discussion.

26. Strategies for Improving CTE Performance Results Understand CTE accountability (CTE students, CTE indicators, how indicators are calculated); Improve data quality – more complete, accurate and reliable data; Help students perform at higher levels; And more… 26

27. Understand CTE Accountability Improve Guidance Documents (published on ODE Web) Conduct Webinar sessions Increased engagement of local CTE leaders 27

28. 28 Data Quality – EMIS data system (PK-12 SLDS) All data for Secondary CTE reported in EMIS Student-level data (student identifier) For FY09, CTE pulled EMIS data from: –Multiple Years -- FY09, FY08, FY07 –Four EMIS reporting periods each year

29. Data Quality -- Strategies for Improving EMIS Data Improve CTE EMIS data verification reports Conduct Webinar sessions and trainings Develop new CTE Data Tools to help Locals –Monitor data while reporting periods are open –Track performance results 29

30. 30 We are conducting Webinars for CTE on different EMIS reporting periods, with expanded information on EMIS reporting and how to use and interpret the CTE EMIS reports.

31. Develop More Capacity to Work with Locals Build more capacity among ODE state staff to understand CTE data and the CTE EMIS data reports. ODE state staff can work one-on-one with local CTE leaders on: –Monitoring local performance data –Tracking local preliminary performance data –Developing Performance Improvement Plans 31

32. Improving Student Performance Focus on Specific CTE Indicators: –Webinar on improving Nontraditional Participation and Nontraditional Completion Focus on Student Sub-groups: –CTE Initiatives on Students With Disabilities 32

33. Local CTE Performance Improvement Initiatives Benchmarking own performance results with other locals. High engagement of staff and students in CTE performance. Using CTE Program performance results as a gauge of program quality and viability. 33

34. What different strategies are you employing to improve CTE performance? 34

35. Perkins IV – “National” CTE Data(?) Much variability in definitions of CTE students and indicators of performance. Much variability in CTE state targets and performance results. Difficult to tell a “National Story” about CTE. 35

36. 36

37. FY2008 CTE Accountability All States and DC, PR Guam, VI (54 total) Indicator State Target Range Actual Performance Range # Met 90% of Target HighLowHighLowYesNo 1S1 97.7322.3097.5525.29484 1S2 96.9711.0097.5011.26475 4S1 95.0050.00100.0042.38511 37

38. Perkins V – Looking ahead… 38 Might we strive for more consistency in student and indicator definitions?

39. Thank you for participating in this discussion on improving CTE performance results! And may your performance results continually improve. Sharon Enright, Ph.D. Ohio Department of Education Office of Career-Technical Education sharon.enright@ode.state.oh.us 614-644-6814 sharon.enright@ode.state.oh.us 39