1. Click to edit Master title style 1 Part II The details

2. Click to edit Master title style 2 Core Indicator Data MIS Data –Data Elements –Funding –Accountability

3. Click to edit Master title style 3 Defining the Data SAM Codes TOP Codes Data Elements Core Indicators –“The Law” –Definitions –Negotiated Performance Targets –Measurement Approaches/Formulas Funding

4. Click to edit Master title style 4 Student Accountability Model (SAM) & Taxonomy of Programs (TOP) Priority “A“ - Apprenticeship –Must have the of the Division of Apprenticeship Standards approval Priority “B“ – Advanced Vocational –Used sparingly, no more than two courses in any one program –“B” level courses must have a “C” prerequisite in the same program area Priority "C" – Clearly Occupational –Generally taken in the middle stage of a program, detracts "drop-ins."

5. Click to edit Master title style 5 Student Accountability Model (SAM) & Taxonomy of Programs (TOP), Continued Priority "D" – Possibly Occupational –Taken by students in the beginning stages of their occupational programs –Can be survey course Priority “E” = Non-Occupational Vocational Flag on TOP code –Designed to identify vocational “Programs” for federal reporting (*) - see Taxonomy of Programs, Sixth Edition, July 2007

6. Click to edit Master title style 6 Data Elements MIS System Students, Courses, Degrees, Services Student VTEA Data Elements –Economically Disadvantaged –Single Parent –Displaced Homemaker –Cooperative Work Experience Education –Tech Prep –Migrant Worker - Implementation in MIS SU 09

7. Click to edit Master title style 7 Accountability Requirements Section 113(b) 5 core indicators of performance: 1.Student attainment of technical skill proficiencies; 2.Student attainment of credential, certificate, or degree; 3.Student retention in postsecondary education or transfer; 4.Student placement in military, apprenticeship, or employment 5.Student participation/completion of non-traditional training State and Local adjusted levels of performance –Levels of performance negotiated with USDE / State Results reported annually

8. Click to edit Master title style 8 Perkins IV (2006) Core Indicators 1.Technical Skill Attainment Successful CTE course completion (GPA) 2.Completions Program completion–Certificate, Degree & Transfer Prepared 3.Persistence & Transfer Student persistence in Higher Ed 4.Placement Placement in apprenticeship, employment, military, fed gov 5.Equity -- Nontraditional Fields Participation (5a)/Completion (5b) - nontraditional “fields”

9. Click to edit Master title style 9 Cohort Definitions Used for Measurement Participant: NT Participation –Perkins III -Any enrollment in a CTE course (TOP) –Perkins IV – concentrator using assigned major Concentrator: All other indicators Cohort of students enrolled during the cohort year and –Successfully completed at least one course in the middle or end of a program (SAM A-C) and 12 vocational units within a single discipline (two digit TOP) or –Program completion as indicated by receipt of ANY vocational credit certificate or degree Leavers: Not enrolled in year following cohort year –2P1 - Completions –4P1 - Placement (Employment)

10. Click to edit Master title style 10 Assigning a Program to a Student 1.Award –TOP code of CTE Certificate or Degree 2.Concentrators Hierarchy based on SAM Priority code Assigned to the TOP where most CTE units occurred

11. Click to edit Master title style 11 Timeline for Outcomes & Outputs Negotiated Performance 2008-09 –Negotiated Spring 2008 –Reports publish in Spring 2009 Cohort Year (2006-07) +1 yr for outcomes (2007-08) –Transfer –Persistence –Employment Outcomes have already occurred –Target low performance now!

12. Click to edit Master title style 12 Timeline for Outcomes & Outputs Program Year 2006-072007-082008-092009-10 Negotiated inFall 2007Spring 2007Fall 2007Spring 2008Fall 2008Spring 2009Fall 2009Spring 2010 Measured inFall 2006 Fall 2007Spring 2008Fall 2008Spring 2009Fall 2009Spring 2010 Outcome Years 2007-08 cohort w/ 2008-09 outcomes 2006-07 cohort w/ 2007-08 outcomes 2005-06 cohort w/ 2006-07 outcomes

13. Click to edit Master title style 13 Core Indicator 1 Technical Skill Attainment All Concentrators Successful Course Completions –Technical Skill Proficiencies Vocational (CTE) Courses –SAM A-C –Vocational TOP –G.P.A. –Grade reports (moved to Data Mart)

14. Click to edit Master title style 14 Core Indicator 1: Measurement & Performance Levels SAM A - C Courses: # Student concentrators with GPA > 2.00 ÷ # Students concentrators with Grades A – F Excludes students with only CR/NC or P/NP grades in SAM A-C courses Plan YearTargetActual* 2005-06 93.65% 2006-07 92.58% 2007-08 92.46% 2008-09 92.46%92.xx% 2009-10 92.yy% % * Based on spring 2008 Perkins IV reports.

15. CI 1 - 2008 Data

16. CI 1 - 2008 Data - Student counts

17. CI 1 - 2008 Data

18. Fake Data Exaggerated 93.67%

19. Fake Data Exaggerated 84.3% 83.2% 92.5%

20. Click to edit Master title style 20 Forecasting Wide range of forecasting techniques –Qualitative Forecasting Methods Judgmental Forecasting –Expert Forecasting –Consensus Forecasting –Informal –Quantitative Forecasting Methods Time Series –Naïve Forecasting –Averaging Causal / Relational Models –assume cause and effect, and cause can be used to predict outcomes –if you know one variable, you can forecast the other Sophisticated

21. Click to edit Master title style 21 Qualitative Forecasting Methods Judgmental Forecasting –Expert Forecasting –Consensus Forecasting –Informal –Work best when background conditions are changing rapidly When economic, political or administrative conditions are in flux, When quantitative methods may not capture important information about factors that are likely to alter historical patterns. (e.g., new large apprenticeship program)

22. Click to edit Master title style 22 Qualitative Forecasting Weaknesses anchoring events –allowing recent events to influence perceptions about future events, e.g. the college hosting a recent institute on student learning information availability –over-weighting the use of readily available information false correlation –incorporating information about factors that are assumed to influence outcomes, but do not inconsistency in methods and judgments –forecasters using different strategies over time to make their judgments, making them less reliable selective perceptions –ignoring important information that conflicts with the forecaster’s view about causal relationships wishful thinking –giving undue weight to what forecasters and government officials would like to see happen group think –when the dynamics of forming a consensus leads individuals to reinforce each other’s views rather than maintaining independent judgments political pressure –where forecasters adjust estimates to meet the imperatives of budgetary or other college constraints.

23. Click to edit Master title style 23 Simple Quantitative Naïve Forecasting –Random Walk Last known Random Walk with drift –Averages –Seasonal adjustments used in expert forecasting as the starting point for estimates that are then adjusted mentally

24. Click to edit Master title style 24 Random Walk Last known

25. Click to edit Master title style 25 Last known Random Walk

26. Click to edit Master title style 26 Random Walk with drift

27. Click to edit Master title style 27 Random Walk with drift

28. Averages: CI 1 - 2008 Data 92.34% Random walk w drift 92.46% Random walk

29. Click to edit Master title style 29 Moving Average moving average –the last N periods of data are used equally –all prior observations are not used Provided in the workbook 3 Yr Ave4 Yr Ave5 Yr Ave Average of rates 92.90%93.11%93.22% Average adding N's 92.90%93.12%93.23%

30. Averages: CI 1 - 2008 Data 92.6%

31. What we used on CI 1 - 2008 Data 92.46%

32. Click to edit Master title style 32 Three Basic “Chuck” Rules With no application of local knowledge or sophisticated projections: –Declining for three or more years random walk, last known –Increasing for three or more years three year average –Increasing and decreasing three year average.

33. Click to edit Master title style 33 Core Indicator 2 Program Completions Completers (numerator) –Transfer Prepared –Award in Current Year –AA/AS degrees –Certificates –Award in subsequent year with no Voc coursework –or Equivalent Leavers & Completers (denominator) –Left system (college) for one year and/or –Award in Current Year –AA/AS degrees –Certificates –Transfer Prepared –Award in subsequent year with no Voc coursework Removed Persisters & Life-Long-Learners

34. Click to edit Master title style 34 CI 2-Completions: Measurement & Performance Levels Certificate/Degree/Transfer Prepared ÷ Concentrators (Leavers & Completers) Not LLL * Based on Perkins IV data. YearTargetActual* 2005-0664.53% 2006-0765.70% 2007-0868.16% 2008-093 yr Ave66.13%xx.xx% 2007-0888.03% 2008-093 yr Ave87.20%xx.xx%

35. CI 2 2008 data 66.13%

36. CI 2 2008 data revised 3 yr ave

37. Click to edit Master title style 37 Core Indicator 3 Persistence & Transfer Concentrators who were not leavers in the year following the cohort year or Transfers to CCC/CSU/UC/Privates (National Student Loan Clearinghouse) ÷ All Concentrators who were not completers with degrees or certificates (unless transferring) YearTargetActual* 2005-0693.02% 2006-0792.80% 2007-0893.03% 2008-093 yr Ave92.95%xx.xx%

38. CI 3 – 2008 data

39. 92.95% 82.95%

40. Click to edit Master title style 40 Core Indicator 4 Placement Placement –Leavers and Completers Minus Continuing in Two or Four Year Institutions – CCC or National Student Loan Clearinghouse –Employment 1 st year out UI wage file match –Employment any quarter in Academic Yr after cohort year –Apprenticeship, Military, Fed Gov

41. Click to edit Master title style 41 CI 4 Placement : Measurement & Performance Levels Leavers & Completers in UI covered employment or Apprenticeship, Military, Fed Gov ÷ All Leavers & Completers *Based on Spring 2008 Perkins IV data. YearTargetActual* 2005-0678.40% 2006-0780.15% 2007-0881.04% 2008-093 yr Ave79.86%xx.xx%

42. CI 4 2008 data 79.86% 71.87%

43. CI 4 2008 data revised 80.91% 72.82%

44. Click to edit Master title style 44 Core Indicator 5 Gender Equity Programs leading to Nontraditional Fields (e.g., Men in Nursing – Women in Auto) 75% / 25% from 2000 census employment data –NAPE developed Nontraditional CIP table Job codes (SOC) mapped to 2000 Census data SOC codes mapped to CIP (USDE) CIP codes mapped to TOP (CCC)

45. Click to edit Master title style 45 Core Indicator 5 Gender Equity Programs leading to Nontraditional Fields Nontraditional Gender Students ÷ All Students in NT Program

46. Click to edit Master title style 46 CI 5a: Participation Measurement & Performance Levels Nontraditional participants enrolled in a Nontraditional TOP Code ÷ All participants enrolled in a Nontraditional TOP Code * Based on Spring 2008 Perkins IV data. YearTargetActual* 2005-0622.06% 2006-0721.78% 2007-0821.47% 2008-09 Random Walk 21.47%xx.xx%

47. CI 5a 2008 data 21.47% 19.32%

48. CI 5a 2008 data revised 21.63% 19.47%

49. Click to edit Master title style 49 Nontraditional “completers” of nontraditional programs ÷ All “completers” of nontraditional programs CI 5b Completion: Measurement & Performance Levels * Based on Spring 2008 Perkins IV data. YearTargetActual* 2005-0623.85% 2006-0723.33% 2007-0823.28% 2008-09 Random Walk 23.28%xx.xx%

50. CI 5b – Spring 2008 data 23.28% 20.95%

51. CI 5b - 2008 data revised 25.38% 22.84%

52. Click to edit Master title style 52 CI & Negotiation Reports Negotiation reports –Developed for negotiations College, District, and State Level Special Pops detail 8 years history Final version 10/14/08 All Core Indicator Reports –Easier access – Excel or PDF reports –Easier graphing in Excel Progressive scrutiny on use of reports –Below 90% of Target in any Indicator

53. Click to edit Master title style 53 Access http://www.cccco.edu Important Documentation –Negotiation Report Instructions (1 st Tab) –Quick Reference (not yet available) –System Documentation –System Requirements No more Installing plug-in No Administrative rights requirements Best viewed with MS Internet Explorer Email notification when available

54. Report Selections

55. Report Years

58. Indicator 1P1

63. misweb

66. Click to edit Master title style 66 Report Structures Negotiation Workbooks –FAUPL negotiation worksheet Perkins IC - Local Application Forms Perkins IC - Local Application –Targets and Performance Perkins IC - Final Report Summary Reports –All five Indicators on one page Answer sheet style Detail Reports with counts Trend Reports –Percents and counts –Detailed breakouts for each Indicator component Special Population Reports

67. Click to edit Master title style 67 Negotiation Workbooks Worksheets: Instructions Counts Percentages Charts –Rates –Counts Data work tables –Percentages, Success, Totals “only” sheets –Fees –Employment

68. Negotiation Tables - Counts

69. Negotiation Tables - Percentages

70. Negotiation Charts

71. Click to edit Master title style 71 Practice using the Data Negotiation Exercise: Review FAUPL Worksheet Walk through common scenarios Complete the FAUPL using first indicators Negotiate or Accept State Targets

72. Faupl

73. Click to edit Master title style 73 Local Application CTE-6

74. Click to edit Master title style 74 2008 FAUPL

75. FAUPL col 3

76. Click to edit Master title style 76 Percentages

77. FAUPL col 456

78. Indicator 1P1 Rates Default Groups

79. Indicator 1P1 2 Rate

80. Indicator 1P1 Success

81. Indicator 1P1 Totals

82. Indicator 1P1

83. Indicator 1P1 2 Rate 83.17% 74.85%

84. Columns 4-5-6

85. Justification

86. Click to edit Master title style 86 Indicator 2P1

87. Click to edit Master title style 87 Indicator 2P1 table

88. Faupl paste

89. Chart 2p1

90. Click to edit Master title style 90 Average

91. Faupl 2p1

92. Faupl 2p1 complete

93. Click to edit Master title style 93 Completing the Negotiation Email the completed FAUPL to Monitor Monitor will either –Accept the proposed level & respond by email –Begin a conversation and negotiation District will enter negotiated levels into the Application.

94. Click to edit Master title style 94 Negotiating Targets State negotiates targets USDE –Targets for 1 year –Next 2 years (3 rd & 4 th ) –Scheduled for April 2009 Worksheets without state targets are available now State Targets will be provided when available Locals either: –Accept state targets –Negotiate local targets Included in Local Plans –Targets for next year –Negotiations complete by May 15

95. Click to edit Master title style 95 The 10,000ft. View 1.District Assesses performance with form I-E-D a.Determine improvement status b.If necessary, alter planning process c.Complete 2009-10 planning 2.Complete the 2009-10 FAUPLE a.Analyze overall district performance b.Determine proposed targets for the next year 3.Negotiate targets with System Office monitor 4.Complete local application form CTE-6 5.Complete local application form CTE-7

96. Click to edit Master title style 96 Questions? Resources: –Project Monitor –Nontraditional Joint Special Populations Committee (JSPAC.org), Institute for Women in Trades, Technology & Science (IWITTS.com), National Alliance for Partnerships in Equity (NAPE.org) –Forecasting literature Fee Impact Study - CCCCO - –http://www.cccco.edu/Portals/4/Reports/impact_study_051.pdfhttp://www.cccco.edu/Portals/4/Reports/impact_study_051.pdf Enrollment Management papers –http://www.cap-esp.com/http://www.cap-esp.com/ Keith Guerin (1999) Chuck Wiseley - cwiseley@cccco.edu, 916.327.5895cwiseley@cccco.edu