CTE Data Reporting Originally presented at OAEP Spring Conference 2017 (with a few modifications/additions) Presenter: Marcy Roll - EMIS Coordinator Upper Valley Career Center Contributors: Cathy Glatz, MCEP, DM META Solutions Rhonda Palmer, CEP META Solutions

CTE reporting – Teamwork is Critical GOOD DATA ISN’T A ONE PERSON JOB – WORK WITH A TEAM Meet with: Superintendent Treasurer Testing Coordinator Program Instructor Program Supervisors Guidance Office Registrar How presentation will benefit audience: Adult learners are more interested in a subject if they know how or why it is important to them. Presenter’s level of expertise in the subject: Briefly state your credentials in this area, or explain why participants should listen to you.

CTE Overview Everything CTE begins with a CTE-26 A CTE-26 is a digital form outlining CTE program characteristics The CTE-26 is submitted for approval by the District/Career Center to ODE Once approval is received, district: Hires certified teacher Enrolls students in approved program Tests students per program requirements Reports student progress in program

CTE Reference Documents – MUST READS General Student Reporting – EMIS Manual CTE Specific Reporting Elements ODE Website>Topics>Career Tech Compliance, Funding & Accountability CTE-26 Process Data & Accountability Career-Technical Education Accountability Manual (FY2015) Resources Program Matrix Career-Tech Assessment System (Assessment Matrix) These two combined for FY2018!!! Hint – I copy to my desktop and add notes as needed. Examples…found missing EMIS subject code for FCS – posted helpdesk ticket comments on matrix. Industry Recognized Credential List Ohio Career-Tech Planning District Map

CTE Specific – Data Collector Manifests & Reports Staff (L) Manifest(s) CTE Funding Reports CTE Adjusted Funding Reports Invalid Certification Reports HQT Reports March (D) Manifest Follow-up survey data for eligible students FYI – I use mail merge to insert student names, SSID, Pathway and phone/email information into the follow up form (pulled from my student software) Assessments and Industry Credentials Assessment (A) Manifest(s) Industry Credentials, Technical Assessments and WorkKeys scores

Staff (L)Manifests and Reports Two (2) Staff and Course Collections – Initial & Final If any of the courses you reported in your (CN) course file have a Curriculum Code beginning with a ‘V’, or a CTE subject code, you will receive Level 2 CTE reports in the Data Collector. (Check EMIS Manual for Subject Codes (Section 4.7) under Section 4 – COURSE RECORDS. There is a Career-Technical Education Section that lists CTE- specific subject codes) FYI – NEW combined EMIS manuals include all sections of the applicable chapter (for instance EMIS Course records includes sections 4.1 through 4.7) Data Collector - Level 2 Reports CTE FTE and CTE Course funding reports (Initial & Final) HQT data (Initial ONLY) Post Processing Reports on ITC EMIS server Invalid Certification Reports (Initial ONLY) Will probably become Data Collector reports (received files) this year (FY18)??

March (D)Follow-up Manifest March Follow-up is a manifest that collects information about students who have completed > 50% of an approved CTE program and have left school. This data is used for your CTE report card. A tool called a ‘Follow-up survey’ is used to gather data on a student’s: Employment and Education status after leaving school Additional information collected in March (D), but not part of survey: CTE Assessments and Industry Credentials Graduation Information CTSO (Career Technical Student Organization) Participation

March (D) Follow-up – Survey Information Timeline is October 1 to December 31, or no later than the close of March (D) reporting The survey asks about Post Program status regarding Apprenticeship, Employment, Military and Post Secondary Education or Advanced Training Hint – I add CTSO participation level information to the form for easier input of data Current year surveys can be found on ODE’s website: Search: ‘Follow-up survey’

March (D)Follow-up Manifest The Process: In early January, ODE will place a ‘CTE March Follow Up (GV) Extract for FFE’ file in the Level 2 section of the March Manifest in the Data Collector. This (GV) file contains students classified as: (1) ‘concentrators’ AND (2) who were reported as leaving school. (concentrator definition on next slide) This file is loaded into a different reporting database called Flat File Editor (FFE). Your ITC can help! Post Program elements are collected on these concentrators using a Follow- up survey Survey data is then entered in FFE for the responding students Survey data is then exported from FFE in (GV) file and uploaded to the Data Collector

March (D)Follow-up – Concentrator Element Who is a ‘Concentrator’? NOTE – must first have been a participant in a workforce development program. A student who meets both of the conditions below is classified as a ‘concentrator’: Successful completion of >50% of an approved CTE program AND: Reported to ODE with an FN Concentrator element (FN290) that is aligned with the pathway code from CTE-26 and Program Matrix

March (D)Follow-up – Concentrator Element Concentrators Drive Everything! Concentrators make up the denominator of your CTE report card Accurate data is critical! Check: Student FN concentrator value vs. their course subject code vs. program matrix and CTE-26 Currently there are NO reports that will point out if there is a mis-alignment – not an easy check, but this is crucial Non-Aligned concentrators will be excluded from CTE Report Card If you use the wrong concentrator value for a student, the student will not be part of CTE Follow-up nor CTE Report Card data Only recourse may be an appeal…but not always granted/permitted

March (D) Follow-up – Leaving School Element Student must have been reported as leaving school as a: Graduate–MUST HAVE DIPLOMA INFORMATION SUBMITTED & withdrawal code = 99 OR With one of the following withdrawal codes: 47, 71, 72, 73, 74, 75, 76, 79

March (D) Follow-up – References Refer to EMIS Manual 2.12 - CTE Workforce Development Follow-up (GV) record Current year surveys can be found on ODE’s website. Search: ‘Follow-up survey’ A helpful best practices document can be found on ODE’s website. Search: ‘Follow-up strategies’ NOTE: Be sure to check the year-end suggestions!

March (D) Follow-up – More Information Work closely with stakeholders The district employing the Teacher of the Workforce Development course reports the March Follow-up (GV) record Let program instructors know surveys have been distributed Follow-up, means just that – follow-up on students in cohort EVERY student matters and can change the letter grade on report card (especially for those with smaller CTE enrollments) Use best practices for follow-up surveys – Non-responses negatively impact report card results Survey may be administered by phone, mail, e-mail, a web-based or social networking system, in person or some other standard survey process

March Follow-up Manifest – Assessments Assessments will count ONLY if aligned with the assessment matrix and reported area of concentration It is critical that the correct area of concentration is reported for a student based upon course subject code AND the correct test is administered and reported!! Any mis-alignment will show up ONLY after the reporting periods have closed Appeals will sometimes help with mis-aligned tests, but better to be accurate when the windows are open!

Assessments Technical Assessments (GY) A technical assessment taken and reported MUST align with the student’s CTE program of concentration. Use CTE Assessment Matrix! All students, including students with a disability, must be assessed if there is an assessment available for the student’s CTE Program Technical Assessments count on the CTE report card Achievement Indicator Check the CTE Assessment Matrix for reference ODE website: Topics>Career Tech>Resources>Career-Tech Assessment System>Assessment Matrix for appropriate year

Assessments Industry Credentials (GW) An industry credential is a credential, certification, or license developed and awarded by an industry association or state licensing agency. Students are not required to get an industry credential, but if a student does receive one of the industry credentials listed in the FA section of the EMIS Manual, the district should report the credential to EMIS. Many students may receive after graduation…such as Cosmetology, so if not previously reported in yearend data, need to add to March Follow Up. Count on the CTE report card if student earns twelve (12) points in one high- demand career field Prepared for Success and Post Program Outcome indicators ALSO count on Traditional public report card – Prepared for Success Indicator if student earns twelve (12) points in one high-demand career field

Assessments WorkKeys (WK) This job skills assessment is a measure of work-force readiness and employability for the purpose of high school graduation This assessment has three sections: Reading Applied Math Locating Information Students in Classes of 2018 and 2019 must earn a total of thirteen (13) points across the three WorkKeys sections to be considered as passing. Must earn at least three (3) points on each section of the test.

Assessments – Best Practices Monitor and verify reports Data Collector Correct Level 1 errors Reports Tab (new this year) All reports, including Level 1, Level 2 and files received from ODE will be located here. Check CTE reports! They are very helpful for verification (and will help avoid ‘surprises’ when report card is released) Report descriptions can be found on ODE’s website. Search: File Descriptions Industry Credentials Report Technical Assessment Participation Report Technical Assessment Passed Report Post Program Placement Report

CTE DATA – What is collected? – Which manifest? See accompanying spreadsheet Course and Staff Data (Both Initial Staff & Final Staff Windows) All Staff & Student course information, Certification, HQT (Certification & HQT Initial Staff Window only) Student Data (Beginning of Year, Mid-Year and Year-End Windows) Concentrators, CBI disadvantagement, etc. Assessment Data (Assessment, Graduation & March Follow-up Windows) Industry Credentials (GW): CTE Assessment, Non-State Assessments for LRC Collection, Graduation and March Collections Technical Assessments (GY): CTE Assessment and March Collections WorkKeys (WK): Non-State Assessments for LRC Collection and Graduation Collection Graduate Data (Graduation Window) Diploma information (Traditional district reports) March (March Follow-up Window) Follow-up survey information along with GW and GY CTE Assessments

CTE Reporting Resources ODE Newsflashes Presentations CTE staff OAEP Conferences usually have CTE Content specific sessions. Check for availability Fall CTE Conference OAEP Spring Conferences ITC Trainings & Tickets Career Tech Planning District Contact Your district staff, along with other area ‘life-line’ EMIS Coordinators

Questions?