Data Warehouse State Reporting Errors Overview All About Errors Data Warehouse State Reporting Errors Overview Jeff Bittner 9/21/2018

Agenda Data Migration from L0 to L2 Accounts & Systems Types of Errors Import Verification L0 to L1 M2L1C Errors M2L1C Warnings Testing Errors Not “Errors” Information Reports Import Messages No Enrollment Reports Exclusion Reports NYSSIS Hold Queue References SED References NERIC References 9/21/2018

Data Migration from 0 to 2 Some Testing Data From the Data Migration Cheat Sheet on the DW NERIC Downloads page. 9/21/2018

When is Data Due? Student data must be uploaded to L1 on Wednesdays in order to refresh on Level2 the following Monday. Staff Data uploaded to L1 will refresh on L2 the following morning. **Only data that does not get caught in any errors will migrate to L2** There are periodic deadlines throughout the year. Read our Listserv emails Read our newsletters Watch our calendar 9/21/2018

Level2 Refresh Schedule Level2 Reports Student data updated every Monday Staff data updated daily (does not include Staff-Student-Course) PD Reports Updated every Monday Certification Button is disabled while the PD is being refreshed Check the Message board. Read the entire message and when it was last updated. TAA ePMF: Staff Snapshot and Staff Assignment data updated nightly while ePMF is open. TSRV: Staff-Student-Course (SSC) data is updated every Monday while TSRV is open. Only courses 3-8 ELA, 3-8 Math, and Math regents courses are in the SSC file and will appear on the TSRV. 9/21/2018

data.nysed.gov Refresh There is a staggered release of data to data.nysed.gov. 2016-2017 data was released in the following phases: Mid August Grades 3–8 English Language Arts (ELA) and mathematics test scores (including scale scores and performance levels) Mid December Enrollment, FRPL, Attendance, Suspensions, Staff, Completers, Non-completers, Post-graduation Plans of Completers, Regents, RCT, Total Cohort, 3-8 ELA/Math, 4 & 8 Science, NYSESLAT, NYSAA, Accountability, and Fiscal Supplements Just released early February 2017 Graduation Rate (2017 grads from SIRS 201 Total Cohort) Still to be released Staff Evaluation, and more… Data is first released as an “Embargoed Release” Embargoed data cannot be discussed at public meetings or released to the public or media. 9/21/2018

Accounts & Systems 9/21/2018

9/21/2018

Types of Errors 9/21/2018

Import Errors What are they? How do I know if I have any? Import errors are errors you get upon import into Level0. They are basic errors. Data with Import Errors does not actually load onto Level0. You can’t push a template with an Import Error to L1. Fix these first. How do I know if I have any? You will see them immediately upon import into Level0. You can also find them under Reports> “pick a template” in Level0. 9/21/2018

OR 9/21/2018

Verification Errors What are they? How do I know if I have any? Verification errors are errors you get on data that has successfully loaded to L0. They are more sophisticated errors. Templates with Verification Errors will not move to L1. They are often comparing data between two templates. Run these and fix these prior to pushing data to L1. How do I know if I have any? Verification Errors are found under Reports> “pick a template”> Run Verification Rpt in Level0. ALWAYS run Verification reports for all the templates you plan on uploading to L1 prior to going to L1 Data Prep. The “Validate Now” button at L1-Data Prep runs Verification checks. You should never have Verification error surprises at L1-Data Prep. 9/21/2018

1 2 3 9/21/2018

Dashboard Report Error Counts and time stamps Verify all data button Import, update, upload dates Red cells 9/21/2018

Level0 to L1 Errors What are they? How do I know if I have any? L0toL1 errors appear the day after you upload a template to L1. Data with L0toL1 errors does not migrate to L1. The most common L0toL1 errors appear if: You push a template to L1 more than once in a single day. You push any data to L1 for a student who does not have a demographics record at L1. You push any data tied to a course and that course has not make it to L1 yet. How do I know if I have any? The DW team will send you an email that a L0toL1 error is in your Serv-U folder. Its very important that you fix the error and push the data again ASAP. L0toL1 error are only generated once (unlike M2L1C errors) so if you don’t take care of it right away, they are easily forgotten. 9/21/2018

After that, they are archived and will not appear again. L0toL1 errors are found in Serv-U in the Warehouse Errors folder only the day after they occur. After that, they are archived and will not appear again. 9/21/2018

Move to Level1 Container (M2L1C) Errors What are they? M2L1C errors are the final error checks before data moves to Level2. They are the highest level errors. Data with M2L1C errors does not migrate to Level2. M2L1C errors appear every night until the data is corrected and submitted to L1. How do I know if I have any? The day after you push data to L1, you should check Serv-U Warehouse Error folder to see if you have any M2L1C errors. If you have any, fix your source, upload new data to L0, and push again to L1. Each Friday, the NERIC DW team will send an email via the Listserv to let you know that the Friday M2L1C errors have been produced. This is the data that did not move on from the container. 9/21/2018

Everyday, errors in the Warehouse Errors folder are archived into the “Archives” folder into a new folder with the date of the errors as the name. 9/21/2018

Student Entry Exit M2L1C Error example: Check your Student M2L1C Errors in this order. Demographics/Student Lite (anyone who is caught here will error in the rest) Enrollment/Entry Exit (anyone who is caught here will error in the rest) Then the rest Check your Staff M2L1C Errors in this order. Staff Snapshot (anyone who is caught here will error in the rest) Then the rest Student Entry Exit M2L1C Error example: 9/21/2018

Common M2L1C Errors The Teach ID entered for this staff member is not valid. Fix the ID on the snapshot and send email to dwhelp@neric.org requesting we delete this incorrect record from L1. The student does not have a NYSSIS ID at this time. Check to make sure they are not caught in your NYSSIS Queue. If they are, make a match. If not, give this a day or two to clear up. If you see this error for more than a few days for the same student, let us know. Any student with this error, will also fail in Enrollment and any other template they are included in. Either this students Demographic record has not been pushed to L1 or the Demographics record had an error and failed to migrate to L1. 9/21/2018

M2L1C Errors - Locations Some M2L1C errors have to do with the BEDS Code: (BEDS codes can change from year to year. Send location requests to dwhelp@neric.org) GD Enrollment can only be at approved locations. Go to SEDREF and find AHSEP location. Go to SEDREF and find the correct BEDS Code. The one you are using is not valid this year. Non-Charter Schools can’t enroll students with a 0011 at a Charter School location. Should likely use the 5905. Must be grade PS if enrolled at a County BEDS code. 9/21/2018

M2L1C WARNINGS What are they? How do I know if I have any? M2L1C WARNINGS are slightly different than M2L1C Errors. In most instances, the data will still move to L1C however not always. It depends on the template and the specific Warning. M2L1C WARNINGS appear every night, just like the Errors. However, some data with WARNINGS is actually correct, so there is nothing to fix. How do I know if I have any? Just like M2L1C Errors, the day after you push data to L1, you should check Serv-U Warehouse Error folder to see if you have any M2L1C WARNINGS. These look the same as and are in the same file as M2L1C errors but the Error Message starts with ****WARNING**** 9/21/2018

Common Student M2L1C WARNINGS Program Service: Data does not move. With no enrollment at L2, the code can’t be tied to a student. Student Grade Detail: Data moves. There is a known ST issue causing this. Daily Attendance: Excused is at top of hierarchy. One attendance code for each student/day combo moves. SpEd Snapshot: Data moves. 9/21/2018

Testing Errors What are they? How do I know if I have any? Any issue that is holding up processing your answer sheets or loading your data to L1. For example: 3-8 exams have an issue with your answer sheets that are holding up the scoring process. No constructed response score. Student not enrolled at the time of test (M2L1C). Regents data can’t be loaded to L1 because of some issue with reported or missing reported data. NERIC has loaded your Response and/or Accommodation files and you have not yet reported a score (August). Student took a test on your answer sheet but has no enrollment. How do I know if I have any? After your answer sheets have been sent to NERIC, your 3-8 Testing Coordinator hear from the NERIC Testing Team regarding an error. (Regents will come from Testing@neric.org , 3-8 will come from Testing38@neric.org . Errors are found on Serv-U in the 3-8 Testing folder (or M2L1C). Email Testing@neric.org with questions regarding these errors or to find out who they have as your 3-8 Testing Coordinator. Regents errors are found on Serv-U in the Warehouse Errors folder. Or, you may hear from DW directly. 9/21/2018

Who Loads Regents Testing Data? Districts only load Regents Scores for August Regents and January Physics. For Jan and June Regents, districts must review ASAP and Level2 Reports to ensure data is correct. If the district has an official score that is different than the one loaded by NERIC, they should wait until the last time the NERIC Testing team loads the data and then load only the differing score(s) to L0 and push to L1. NERIC loads January (except Physics) and June Regents score data directly to L1. Fact (Scores) Accommodation (Those used during test) Response (Answers and points earned) August DISTRICT loads into L0 and push to L1. January June 9/21/2018

Not “Errors” – But Still Important. 9/21/2018

Information Reports What are they? How do I know if I have any? Information Reports are just that, informational. They are designed to alert you to potential data issues, however are they not errors. Do not stop data from moving to L1. How do I know if I have any? In Level0 go to Reports > Information Rprts > Select a Data Type from dropdown > click “Run Report” 9/21/2018

1 Information Reports The capitalized templates have Information Reports 3 2 9/21/2018

Information Reports Tips Program Fact FRPL check based on other data. Looks at other data to see if they student should probably have a FRPL records also. S.E. Snapshot checks based on other data. Looks to see if a student was enrolled on BEDS Day with an active disability record but no BEDS Day Snapshot. Enrollment Ungraded student has no Disability. Staff Assignment Principal must be assigned to every building with enrollment. Staff Student Course Zero course duration Zero Attendance minutes Zero Enrollment minutes More SSC Information Errors on next slides. 9/21/2018

ELA: 5080/5080 = enrolled for entire course duration. Example of SR 7731: Overlapping enrollment for student in same class with same teacher. Notice how both records tied to same State Course Code, start and end dates match, and both have same reporting date. Since the course, teacher, and location are the same, and the enrollment dates overlap - the minutes will be added to calculate what percentage of the course the student was enrolled. In both courses, Enrollment/ Course duration = enrolled for entire course duration. ELA: 5080/5080 = enrolled for entire course duration. Writing: 3000/3000 = enrolled for entire course duration. When minutes are added, Enrollment / Course duration still shows them enrolled for entire course. 8080/8080=1 So, not a problem. The ratio works out to be the same. 9/21/2018

SR7730: Same End Date/Start Date for Course. Notice how both records are tied to same State Course Code, start and end dates are on the same day, and reporting dates are the same. Since the course, teacher, and location are the same, and the enrollment dates overlap - the minutes will be added to calculate what percentage of the course the student was enrolled. The student is enrolled in each class for different amounts of time. Math 7: 704/4570 = enrolled for about 15% of course duration. Math 7a: 4623/5421 = enrolled for about 85% of course duration. When minutes are added, Enrollment / Course duration, the ratio changes. 5327/9991 = enrolled for about 53% or course duration. This is a problem. Students must be enrolled for 60% or more of course duration in order to be used in Teachers Student Growth calculation. 9/21/2018

Import Messages What are they? How do I know if I have any? Import Messages are the details of “Skipped Records” upon import onto L0. Data in the Import Messages did not make it onto L0. How do I know if I have any? After clicking “Validate Data” when importing a file, you will see something like the below. 9/21/2018

DataFlow subscribers will see “Auto-Import” as the Import type. Import Message Log can also be seen by going to Reports > pick a template > and selecting Import Message Log DataFlow subscribers will see “Auto-Import” as the Import type. 9/21/2018

Common Import Message Log This Program Fact record was skipped since 51808 is not a recognized program code on L0 at this time. In this example, 510808 is a CTE code, which have not yet been loaded to L0 for the 17-18 school year. Once loaded, this message will not appear again. NERIC loads CTE codes in the spring. This Course Instructor Assignment record was skipped since the record had a blank Teach ID. For CIA files loaded into L0 via Electronic export, the details of what record was skipped are listed in the Import Message. That functionality is being developed for DataFlow customers. 9/21/2018

No Enrollment Report What are they? How do I know if I have any? No Enrollment Reports are found in L0 and show you if a student has data, but has no enrollment on L0. In order for any student data to make it to L2, they must have an enrollment record. Common reason you may have some: If you reported a student to L2 that should not have been reported, you should make their Demographics record inactive, and delete all other data for that student from L0. Push demo, enrollment, and any other templates you removed data from to L1. This student will appear on the Demographics No Enrollment report for the remainder of the school year. How do I know if I have any? On Level0 go to Reports > pick a template > No Enrollment Rpt The following templates have a no Enrollment report: Demographics, Program Fact, Assessment, Accommodation, S.E. Event, S.E. Snapshot, SSC, Student Class Entry Exit, Student Class Grade Detail, Credit GPA, Student Daily Attendance. 9/21/2018

Exclusion Report on the PD System What are they? The Exclusion Report on the PD system show student who are missing one of the requirements of being included on the VR reports. For example: they have enrollment and a disability at Level2, but no Snapshot. How do I know if I have any? Go to the PD System and select the exclusion Report from the dropdown. It does not matter what VR you choose, just make sure you are correctly in the Oct or EOY reports. 9/21/2018

9/21/2018

NYSSIS Queue Hold Queue What are they? Students in the NYSSIS Hold Queue were not matched up to a NYSSIS ID. They are placed in a hold queue and must be either matched to an existing NYSSIS or a new NYSSIS must be created. Data for the students in the Hold Queue will not move to L2. You may see M2L1C errors like “State Student ID is Blank” for these students until an NYSSIS ID is assigned. How do I know if I have any? Someone in your district is the NYSSIS contact and will receive an automated email from SEDNYSSIS@nysed.gov with the subject “NYSSIS Hold Queue exists for “YOUR” CSD. # Student(s) in Queue.” A link to the NYSSIS System will be included. NYSSIS log in uses your SEDDAS credentials. 9/21/2018

9/21/2018

9/21/2018

9/21/2018

References DW site: http://neric.org/ServiceGuide/DataWarehouse.cfm Flow of data Accounts and systems New Level0 Location Requests Level0 Validation: http://www.p12.nysed.gov/irs/vendors/home.html SIRS Manual: http://www.p12.nysed.gov/irs/sirs/ NYSED Teacher/Staff Data: http://www.p12.nysed.gov/irs/beds/PMF/home.html NYSSIS Guide: http://www.p12.nysed.gov/irs/nyssis/ NYSED Business Portal: https://portal.nysed.gov/abp PD System NYSSIS Instructions for 17-18 VR reports on PD: http://www.p12.nysed.gov/sedcar/forms/instructions/instructions1718.html SEDREF: https://portal.nysed.gov/pls/sedrefpublic/SED.sed_inst_qry_vw$.startup Approved AHSEP programs: http://www.p12.nysed.gov/sss/ssae/AltEd/ 9/21/2018

Error Resolution Tips The Error Message The Error Code Not always super helpful but more often then not gives a clue to where to look to fix the issue. The Error Code Search the L0 Validation document for the code. This will often give you more information about the error. Where in the Data Migration Process did the data get caught? If you get a M2L1C error saying the student has not demo record, don’t look to L0 for a demo record. The demo record is missing from L2. Check the SIRS manual: Table of reporting responsibility Program Fact section give more details who gets the code and how its reported. 9/21/2018

More Error Resolution Tips Data/Template specific guides and trainings SED gives really great guidance on specific reporting like Staff and Special Education. SIRS Manual is a great place to start with all error resolution. The DW downloads page have cheat sheets and guide on specific topics like: Pre-K reporting Reporting students in GED programs Course Instructor Assignment and Class Entry Exit interaction NYSAA ELL/MLL Home Schooled students Summer School Check for errors regularly The longer you wait to review errors the more you will have and the more time that has passes since the data was originally entered. Our suggestion would be to report data weekly to L1. Always make your fixes in the source system Work staff who entered the data incorrectly so the same mistake is not made again. You can always reach out to us at 518-862-5409 or email dwhelp@neric.org 9/21/2018

Thank you 9/21/2018

9/21/2018