1. Perkins Accountability ~ Secondary 2016 CTE Fall Webinar Series
MDE host or Denise
Your phone line is muted. If you have a question, type it in the chat box and we will respond to your questions throughout the webinar.
Again, your phone line is muted, type all questions in the chat section and refer to the PowerPoint side number when possible.

2. Chat, Comment, React MDE host or Denise
Your phone lines are muted to eliminate feedback and background noise.
Use the chat box to ask your questions and comments throughout the webinar. Also look in the chat box for links and resources related to this topic.

3. Minnesota Department of Education
Today’s Presenters
Minnesota Department of Education
Kari-Ann Ediger Results Measurement Specialist
Michelle Kamenov Career Development & Career Technical Education Supervisor
MDE host or Denise

4. Goals Submitting the P-file Perkins Indicator Definitions
Negotiating Local Performance Targets
Summary Reports
Michelle

5. MDE host or Denise: Michelle, what is the P-File?
It is important to understand that the data in the P-file primarily represent course enrollment records. Therefore it is critical that each record contain the appropriate Program and Course code information.
The P-file data are linked with the program and course codes from Table C as well as pre-existing Program Approvals for each district as listed in the Program Approval database. (slide 5) The best way to ensure that the appropriate program and course codes are listed in your P-file is to justify the program and course code combinations listed within the P-file with the program and course codes listed within the Program Approval Database.
The Program Approval Database contains all the program and course codes which have been approved for each given district. Having an approved program means that an appropriately licensed CTE teacher has implemented the course. Only those programs and courses which have been approved need to be listed within the P-file. However, the caveat here is that IF your district has implemented a new course under and existing approved program, then, please send a course update to us here at MDE and we will expedite approval for that course so that you can include it in your P-file submission.

6. The P-File: Data Submission
Submit to MDE via Perkins Website /CarlPerkins/
One person per district is authorized to submit P-file
Contact MDE for directions to identify district personnel to upload data file
Submit P-file by DATE - No extensions granted
MDE host or Denise: When and how does a district submit the P-File
Kari-Ann: Always be sure that any program and course updates have been sent in to MDE. All program and course offerings need to be current within the Program Approval Database.

7. Perkins Indicators Participant 100-239 hours in one career field
Concentrator
240 hours or more in one career field
Secondary Completer
A Concentrator who graduates
MDE host or Denise: Let’s talk about Perkins Indicators. How are they measured?
Michelle:
Although the Perkins law specifies the topical areas for which local consortia are to be held accountable, the state negotiates with the federal agency in order to specify the exact definition of each Performance Indicator.
These definitions describe the “working” definitions of each indicator and are important relative to interpreting the outcome data within each report.

8. 1S1 Academic Attainment in Reading/Language Arts*
Number of 12th grade Participants and Concentrators who have met or exceeded the proficiency level on the high school Reading Assessment (MCA/MTAS)
Number of 12th grade Participants and Concentrators who took the MCA/MTAS and whose high school Reading assessment scores were included in the State’s computation of AYP.
Numerator
_________________________________________
Denominator

9. 1S2 Academic Attainment in Mathematics*
Number of 12th grade Participants and Concentrators who have met or exceeded the proficiency level on the high school Mathematics Assessment (MCA/MTAS)
Number of 12th grade Participants and Concentrators who took the MCA/MTAS and whose high school Mathematics assessment scores were included in the State’s computation of AYP.
Numerator
_________________________________________
Denominator

10. 2S1 Technical Skills Attainment*
Number of Secondary CTE Concentrators within Programs of Study selected by the state for which state-approved technical skill assessment instruments have been identified who pass any of those identified technical skill assessment instruments pertinent to the Program of Study in which concentrator status is obtained.
Number of Secondary CTE Concentrators within Programs of Study selected by the state for which state-approved technical skill assessment instruments have been identified who attempt any of those identified technical skill assessment instruments pertinent to the Program of Study in which Concentrator status is obtained.
FY11 was the first year of reporting TSA outcomes for Secondary. (Prior to FY11, the number of concentrators passing 2 credits within one career field was reported.)
*FY14 and going forward, 2S1 is reported as a single-year analysis
Assessments have been added each subsequent year. State-Approved Technical Skill Assessments have now been identified in 60 of the 79 career pathways. 
Numerator
______________________________________________
Denominator
Kari-Ann
Having a good understanding of these definitions is essential relative to interpreting the results. Understanding the parameters of the data impacts one’s ability to make decisions around program improvement, professional development needs, and yearly plan development, as well as target setting negotiations and strategic funding priorities.

11. 3S1 School Completion
Number of CTE Concentrators who earned a regular secondary school diploma during the reporting year.
Number of CTE Concentrators who left secondary education during the reporting year.
Numerator
______________________________________________
Denominator

12. 4S1 Student Graduation Rates 4-Year Cohort from Grade 9
4S1 Student Graduation Rates 4-Year Cohort from Grade 9 *Prior Year Data*
Number of CTE Concentrators who, in the reporting year, were included as graduated in the State’s computation of its graduation rate as described in Section 1111(b)(2)(C)(vi) of the ESEA.
Number of CTE Concentrators who, in the reporting year, were included in the State’s computation of its graduation rate as defined in the State’s Consolidated Accountability Plan pursuant to Section 1111(b)(2)(C)(vi) of the ESEA.
Includes: 1) graduates, 2) dropouts, 3) *unknowns, and 4) *continuing
*The CTE calculation is the same as the state's 4-year graduation rate. All four categories of students are included in the denominator, including the 1) unknown, and 2) continuing students.
Numerator
___________________________________________
Denominator
Kari-Ann

13. 5S1 Placement (Prior Year Data)
Number of Secondary CTE Completers who register as participating in higher education in a data match with the Statewide Longitudinal Data System (SLDS)
Number of Secondary CTE Completers.
     *Note second quarter enrollment following graduation (October-December). So for the December CAR report we will be reporting on students who graduated at the end of the SY  
Numerator
_________________________________________
Denominator

14. 6S1 Nontraditional Participation
Number of CTE students from underrepresented gender groups who enrolled in a nontraditional CTE course during the current reporting year.
Number of CTE students who enrolled in a nontraditional CTE course during the current reporting year.  
Numerator
_________________________________________
Denominator

15. 6S2 Nontraditional Completion
Number of CTE concentrators from underrepresented gender groups who completed a program that leads to employment in nontraditional fields during the reporting year.
Number of CTE concentrators who completed a program that leads to employment in nontraditional fields during the reporting year.
Numerator
_________________________________________
Denominator
Kari-Ann

16. State and Local Negotiations
Federal/State Negotiations
State/Local Consortia Negotiations
Evaluation of State Performance
Improvement Plans
MDE host or Denise: Each of these indicators has targets attached to them. Can a consortia change their target measurements?
Michelle: The process of negotiating performance levels for each performance indicator and evaluating annual performance against those targets is similar between the state and local consortium.
Although as a state we are considered to have “met” a Performance Target if we fall within 90%; consortium are slightly different. Each consortium is considered to be on Report if they achieve a performance level between 90-99% of an agreed upon Target. Also, if a consortium falls below 90% of the agreed upon Target the consortium is then considered to be on Improvement status for the indicators.
Michelle: Both the Report and the Improvement status mean that a consortium must complete additional information within their Annual Plan. If a consortium’s Actual performance falls within the Improvement status, then, the consortium must document strategies for improvement within their Annual Plan. Further, if a consortium’s Actual performance falls within an Improvement status for 3 years or more, then, they are encouraged to seek technical assistance in order to identify and develop a strategic plan for overcoming any barriers to making continuous improvement.
Evaluation of Local Performance Improvement Plans

17. State/Local Negotiation Process for FY 2017 Targets
Target Time Frame
Accountability Webinars: Overview (1pm)
September 22, 2016
Development of State Proposed Targets
October 2016
Distribute Proposed Targets to Consortia
early November 2016
Consortia Responses Due
late November 2016
Initial State Response
early December 2016
Negotiations Completed
December 2016
Evaluation of Performance
January 2017
Michelle
Negotiation of local Performance Targets occurs annually each year around December.

18. Negotiating Your Targets
Review the proposed targets
Examine your Consortia & District level data
– trends/preliminary data if available
Discuss the targets with your partners/constituents
– have there been any changes in policy or practice that may have affected performance outcomes in the coming year?
Understand the context and rationale for the state proposed targets
Expectation of continuous improvement at the state and local level
Need for the state to be able to meet state targets negotiated with OCTAE
Michelle: Targets set for each consortium are evaluated the following year in order to determine whether a consortium is making continuous improvement. Therefore, engaging in target negotiations with the state is a critical part of the accountability process.
Every year, the state undergoes a thorough review of the trend data for each Performance Indicator and for each consortium in order to propose fair, meaningful, and rigorous yet also achievable targets. These target proposals are then shared forward to consortium leaders. At that point, consortium leaders can respond with a counter-proposal target(s) along with providing a rationale for the counter proposal; or, they may choose to accept the proposed target(s). For example, there are any number of events and/or circumstances which may prevent a consortium from achieving a target if the proposed target is deemed too high. It is important to communicate any programmatic shifts and changes to the state as the state may not always be aware of local context when proposing targets.
Once the state received the information back from the consortium leader (either the accepted target or the counter-proposal), the state then reviews the information taking into consideration any rationale that was provided. Following this exchange, final targets are then determined by the state and communicated back to the consortium.

19. Negotiating Your Targets
If you DO accept the state proposals:
Return the FY15-16 Negotiated Performance form
Include BOTH secondary AND postsecondary signatures
Determine if you will accept the proposed targets, or, propose an alternative target
If you DO NOT accept the state proposals:
Notify the state
Submit alternative target(s) along with the rationale for the proposed alternative(s)
Michelle

20. Negotiating Your Targets
If submitting a counter proposal, include:  
The state proposal and your alternative proposed target
Trends in past performance
Rationale supporting the counter proposal, including any additional relevant data
The rationale must document the context of what is driving observed trends and factors that impact future performance.
New or closed programs
TSA’s added, removed, or changed
Shifts in enrollment and the impact on programs
Changes in policies or practices that may impact program performance or enrollment
Michelle: Keep in mind that in reviewing counter proposals the state must consider the ability of the state to meet their negotiated target (i.e., the ability of the other consortia to make up the difference).

21. Requirements for State and Local Improvement Plans
The language in the Act regarding improvement plans and the withholding of funds is very similar for states and local recipients.
… fail to meet at least 90 percent of an agreed upon State adjusted level of performance for any of the core indicators …
… develop and implement a program improvement plan (with special consideration to performance gaps within subgroups) …
Michelle

22. Kari-Ann
There are 11 accountability reports available from the state. Each of these reports is available at the state, consortium, district, and school level.
Historically consortium reports were mailed out (hard copy) to each consortium leader; if a consortium leader wanted a copy of a district report they needed to contact the authorized user at each district who had access to download their district reports then share them with their consortium leader.
However, beginning in FY16 these reports will be available online within MDE’s Secure Reports site. MDE will be working with consortia leaders in order to help them gain authorization which will allow them to access not only their consortium reports, but also the district reports for each of the districts within their consortium. These reports are available to download in both excel and pdf formats.
In addition to the Secondary reports, which are specific to each consortium and associated district, two additional Status and Trend reports are shared with consortia leaders. The Status report displays whether a consortium met their agreed upon target level of performance, or whether they are on a report or improvement status. The Trend report contains the same information across the last 3 years.

23. Accountability: Making Connections
Kari-Ann
School and state administrators, CTE Consortia Leaders, and CTE Teachers alike are all looking at their Perkins data now more than ever. This is a Good Thing!
Program folks are connecting with their authorized data-submitters and we are seeing a lot of positive impact. At least within a couple of consortia we have heard that principals get together and review their Perkins data and make plans for next steps. There is real value in using these reports to get ahead of the curve and plan for growth and program expansion. It can also be very encouraging to literally “see” the direct benefits that CTE programs have on students’ lives and outcomes.
Reviewing your Perkins data is a critical aspect in helping teachers understand what is going well and where they are really making a difference in their students’ lives in addition to strategically planning for professional development needs and identifying areas for collaboration and improvement.
We will be working closely with Consortia Leaders in the coming months to ensure that everyone has access to their reports online. We also anticipate offering a webinars across the school year in order to assist consortia leaders in making the most of their Perkins data. So, stay tuned!

24. www.cte.mnscu.edu/professionaldevelopment /monthly-webinars.html
CTE Professional Development Resources
/monthly-webinars.html
Webinar Recordings, PowerPoints
Schedule for Fall Webinar Series – 9am
September 29 – Postsecondary Accountability
Date TBD – Secondary Treatment of Money
Calendar of Monthly Webinars – Thursdays, 9am
MDE host or Denise

25. Perkins Fiscal Considerations ~ Postsecondary
Kari-Ann Ediger Results Measurement Specialist Michelle Kamenov Career Development & Career Technical Education Supervisor
Please feel free to call or write Kari-Ann Ediger any time there are accountability, program improvement, data submission, general program approval or other data related questions. I am happy to help in any way that I am able. Debra Blahosky can also assist with data submission questions and program approval database questions.
Enter your questions in the “Chat” field OR request to unmute your phone line and ask your questions.