PIMS Submission Tips for Making PIMS Less Painful Tips and Tricks from PIMS Application Support Amber Lind PIMS Application Support Supervisor

Tools to Make PIMS Less Painful PIMS Resources on the PDE Website Data Collection Calendar PIMS Documents Deletions and the Delete Utility Validation, DQE, and ETL Errors New DQE Rules for 16-17 PIMS Reports - Cognos Tools to Make PIMS Less Painful

PIMS Resources on the PDE Website www.education.pa.gov > Teachers and Administrators > PIMS

www.education.pa.gov > Teachers and Administrators > PIMS

Put in an online PIMS Support Ticket Log into MyPDESuite Additional PIMS Docs Put in an online PIMS Support Ticket

PIMS Webinar Recordings and Slides PIMS Q&A Schedule and Zoom Meeting link 2017 dates are now on the PDE website PIMS Manuals

How To Guides with Detailed information on each collection Excel Reporting Tools for PIMS Submissions

Excel Reporting Tools for PIMS Submissions (continued) Excel Reporting Tools for PIMS Submissions 16-17 SY PIMS Calendar 15-16 SY PIMS Calendars

Data (PIMS) Collection Calendar Elementary-Secondary Data Collection Calendar 2016-2017 v3.1.xlsx

Using the Collection Calendar All PIMS data is due and collections will close by 12:00 PM (Noon) on the final date specified in the Collection Window Field unless otherwise noted Collection Window PIMS available for uploading Follow-Up, Review & Editing Window Collection unavailable, NOT able to upload PDE reviewing data for possible errors – so should you! Correction Window Using the Collection Calendar

Using the Collection Calendar Internal Snapshots Listed on the Calendar under Collection 6 NOT in Chronological order with Collections 1-5 Student, School Enrollment, and Programs Fact templates must be uploaded prior to NOON on the due date. NO Overrides or Correction periods PIMS will be closed (no collections available) while an internal snapshot is being pulled by PDE. Presnapshot and Preliminary reports should be run PRIOR to NOON on the due date in order to confirm data is accurate Using the Collection Calendar

Using the Collection Calendar Data Collection Calendar now has multiple sheets Calendar Detailed Description of each collection including due dates and templates Executive Summary Summary of collection due dates (no templates) ACS Summary Identifies each ACS, with Cognos Path, and Due Date PIMS Data Refresh Schedule Lists the Cognos Refresh Times and Sandbox Refresh Dates Using the Collection Calendar

PIMS Documents www.education.pa.gov > Teachers & Administrators > PIMS > PIMS Documents

Additional PIMS Docs

PIMS Documents Career and Technical Education PDE Data Policy Adult and Secondary CTE Student Follow-Up Survey - 2016 Adult and Secondary Webinar Training Documents - 2015-16 PDE Data Policy PIMS Changes (15-16 SY) PIMS Databases (for LEAs without SIS) PIMS How To Guides and Training Documents PIMS Manuals 2016-2017 Proposed PIMS Changes PIMS Documents

PIMS Documents PIMS HOW TO GUIDES AND TRAINING DOCUMENTS Cognos Report Primer (PDF) Tips for efficiently running reports Guide to Manipulating Data (PDF) How to open CSV file in Excel without losing PIMS formatting Should only be used as a last resort Data must also be corrected in source system, otherwise this will cause errors in future PIMS uploads PIMS Data Maintenance Request - A (PDF) PIMS Data Maintenance Request - B (PDF) PIMS Delete Utility (PDF) PIMS Documents PIMS HOW TO GUIDES AND TRAINING DOCUMENTS

PAsecureID Documents www.education.pa.gov > Teachers & Administrators > PAsecureID

PAsecureID Documentation

PAsecureID Documentation

PAsecureID Documentation Introduction To The Redesigned User Interface (PDF) Screen shots showing how to use PAsecureID Features Creating a Student Batch File (PDF) Steps to make a PAsecureID Batch File in Excel When updating an existing PAsecureID using the “Enter Online” form, enter the student’s PAsecureID to prevent a duplicate from being created Near Match may not appear if not changing Name or DOB PAsecureID Documentation

PIMS Deletions Data Maintenance Form PIMS Delete Utility School Enrollment DELETE

PIMS Data Maintenance Forms can be downloaded from the PDE website www.education.pa.gov Teachers & Administrators PIMS “PIMS Documents” Submit an Online Ticket and submit the form when you get the ticket number in your email Or, call PIMS Support for a ticket number and form PIMS Support may still contact you if we have a question about a deletion request submitted online Only 10 individual can be deleted at a time. More than 10 will result in the deletion of the entire template. Requesting Deletions

PIMS Admins who have completed the Experienced PIMS Data Quality Curriculum (DQC) track are able to do limited deletions in Production PIMS PIMS Delete Utility will delete the entire template PIMS Delete Utility is not available for every collection/template Deletion occurs immediately No need to wait for PDE to process Data Maintenance Form or contact PIMS Support Review the “PIMS Delete Utility (PDF)” from PIMS Documents for steps to use the Delete Utility PIMS Delete Utility

Deletions from School Enrollment Template School Enrollment records can no longer be deleted using the Data Maintenance Form Incorrect School Enrollment record must be uploaded with DELETE in Field 8 Correct School Enrollment record must be included in the file School Enrollment “DELETE”s occur immediately Use the “Student ID Search - School Enrollment Template” report to see which records are currently in the system for a single student Public Folders ‎> eScholar Framework for Cognos - Verify ‎> Validation Reports ‎> Template Verification Deletions from School Enrollment Template

Deleting from School Enrollment Template School Enrollment Template Details Report - Records currently in PIMS warehouse: District Code 1 Location Code 2 School Year Date 3 Student ID 4 Activity Start Date 5 Enrollment Date 6 Enroll Code 7 Enrollment Description (DELETE) 8 Grade Level 9 Residence Status Code 10 123456789 1234 2016-06-30 9876543210 2016-09-01 E01 010 A 2016-11-06 WD01 If student was incorrectly withdrawn as a dropout (WD01) and should be a transfer (WD02), then the WD01 record will need to be deleted. The original WD01 record needs to be included in file with DELETE in Field 8. The correct WD02 record also needs to be included in the file. When changing a withdraw record, you also need to upload the Student template to update the Graduation Status Code (field 65). School Enrollment template for uploading should look like: District Code 1 Location Code 2 School Year Date 3 Student ID 4 Activity Start Date 5 Enrollment Date 6 Enroll Code 7 Enrollment Description (DELETE) 8 Grade Level 9 Residence Status Code 10 123456789 1234 2016-06-30 9876543210 2016-09-01 E01 010 A 2016-11-06 WD01 DELETE WD02

PIMS Submission Errors PIMS Application Home Page Validation Errors (File Manager) Data Quality Engine Errors (DQE) ETL Errors (Batch Manager) - Extract, Transfer and Load

PIMS & Sandbox Home Page Identifies if you are in PIMS or the Sandbox System Messages alert you to changes or downtime In the Sandbox, the message usually lists the next anticipated refresh from Production PIMS Sandbox will be unavailable while being refreshed List of available collections List of which templates and snapshot dates associated with each collection List of the File and Batch Manager DQE rules PIMS & Sandbox Home Page

1 2 3 4 5

Validation Errors in File Manager Errors will be highlighted in Red

Validation Errors in File Manager Validation Errors occur in File Manager To see errors Click on “View Errors” button Find the “error.zip” file and click on “View Error File” button Errors will be highlighted in Red Hover over the Red to see the Error message Errors are usually with valid values For example, reporting Grade Level as 1 instead of 001 Compound Data Quality Engine rules compare fields within a single template For example, Student coded with Special Ed Status (field 38) of Y, but Challenge Type (field 34) is blank Validation Errors in File Manager

Validation Errors in File Manager 123456789_STUDENT_201612090700.TAB Validation Errors in File Manager

Validation Errors in File Manager 123456789_STUDENT_201612090700.TAB FILE DETAILS – FILE ID: 1911753; FILE NAME: 123456789_STUDENT_201612090700.TAB Validation Errors in File Manager

Validation Errors in File Manager The Y value in Field 38 is NOT an invalid value, even though it is in Red. The error is because the combination is incorrect.

Validation Errors in File Manager

Data Quality Engine (DQE) Errors in Batch Manager Errors are Identified in the Batch Details under the “Data Quality Engine Results”

DQE Errors in Batch Manager Batch will have a status of “DQE Complete” with a Red X in the Data Status To see the errors: Click on the “View Errors” button for the Batch Find the Red Xs under “DATA QUALITY ENGINE RESULTS” Click on “View Errors” for the selected Data Quality Check DQE Errors cause ALL the files in the batch to fail Nothing in that batch was submitted to PIMS, so everything must be reuploaded after corrections are made DQE Errors in Batch Manager

Find the validations with Red X under “DATA QUALITY ENGINE RESULTS” Then, click on the “View Errors” button to see the errors.

PIMS and PAsecureID are now cross-referencing: Student First Name (first 3 letters) Student Last Name Date of Birth Gender If multiple discrepancies are identified, the first name is checked in its entirety so LEAs can verify all information for this student is correct. If data is wrong in PAsecureID, correct and then wait at least an hour before uploading again into PIMS DQE Errors in Batch Manager Validation UID Student First(3)Name/Last Name/DOB/Gender

DQE Errors in Batch Manager 1 Student Record for Student ID:########## Last Name in the Student File:CONNOR does not match the Last Name in PA Secure ID:Conner  Student File last name ends in OR, but PAsecureID ends in ER 2 Student Record for Student ID:########## Gender in the Student File:F does not match the Gender in PA Secure ID:M  Student File Gender is F, but PAsecureID has M 3 Student Record for Student ID:########## First Name in the Student File:JONATHAN does not match the First Name in PA Secure ID:JOHNATHAN  Student File first name begins with JON, but PAsecureID starts with JOH 4 Student Record for Student ID:########## Date of Birth in the Student File:2002-02-05 does not match the Date of Birth in PA Secure ID:2005-02-05 ; First Name in the Student File:CHRIS does not match the First Name in PA Secure ID:CHRISTIAN  Student File had DOB of 2002-02-05, but PAsecureID has 2005-02-05 (different year) Student File first name is CHRIS, but PAsecureID has CHRISTIAN* *If multiple discrepancies are identified, the first name is checked entirety (not just first 3 letters) DQE Errors in Batch Manager

New PIMS DQE Checks for 16-17 DQE is now checking PPID/Date Of Birth in TIMS PPID needs to be valid for all professional Staff SSN should no longer be reported to PIMS Professional Staff who are not certified still need a PPID in order to be reported to PIMS Date of Birth on Staff and Staff Snapshot templates must match TIMS DOB must be updated in TIMS if incorrect Data Exceptions only granted for terminated staff with incorrect DOB New PIMS DQE Checks for 16-17

ETL Errors in Batch Manager Extract, Transfer and Load Error are identified with an X or E in the ERROR or ALERT file

ETL Errors in Batch Manager Batch will have a status of “Complete” with a Red X in the Data Status To see the errors: Click on the “View Errors” button for the Batch Find the Red Xs under BATCH FILES Note: DATA QUALITY ENGINE RESULTS will have have Green Checks and DATA QUALITY ENGINE STATUS will say “Successful”, but there are still errors Click on “View Errors” for the file with errors Under ETL GENERATED FILES, find the ERROR files and click ”View File” ETL Errors only stop the individual records with errors. All other records are submitted to PIMS Good Errors ETL Errors in Batch Manager

If the Data Status icon shows a Red X for any of the files, then there are still errors in Batch Manager after the DQE Checks. When the Data Quality Engine Status says Successful, that is referring to the DQE Quality Checks.

If ERROR_SOURCEFILE_MSG has a record count of 0, it can be disregarded The “Load” files can also be disregarded, they are never Error files

ETL Errors in Batch Manager This Student Status (field 67) Error is occurring because many valid values from 15-16 SY were retired for 16-17 SY. See Appendix L for a list of valid values. Errors are noted with an X in one of the columns Or, E if in ALERT_VALID_VALUES file Hyphens (-) are place holders, not errors Errors are NOT highlighted Red Red Highlights are only used in File Manager ETL Errors in Batch Manager

PIMS Reports (Cognos) eScholar Framework for Cognos – Verify eScholar Framework for Cognos – Production

Verify & Validation Reports Verify reports update Immediately Use to verify individual records that were submitted to PIMS Verify & Validation Reports

Verify & Validation Reports

Template Verification Reports These reports show Individual Records uploaded on Each Template

Each PIMS Template has a “Template Details” Report for verifying records

Cognos Defaults to showing 15 reports at a Time Use the “Next Page” button to see Additional Reports

Or change the Default number of lines from “My Preferences”

Personalizing “My Preferences” Change the “Number of entries in list view” to 100 “Separators in List view” option makes it easier to match up each report with the Action icons Personalizing “My Preferences”

Useful Validation Reports: Presnapshot Reports (generic)

Presnapshot Verification Reports can be run for any Snapshot Date These reports do NOT exclude students based on Assessment Type (fields 212, 214-216) or LEP/ELL Status (field 41) Presnap Verification Report -4- Students to be Excluded from Snapshot Shows students who are EXCLUDED from an Internal Snapshot and the withdraw record causing the exclusion Presnapshot reports

Each PIMS “Data Set” has at least one folder in Production Accuracy Certification Statements (ACS) are found in Production The system must go through Refresh to see current data Noon 5am Refresh usually last around 2 hours Production Reports

Production Reports

Production Reports Use Next Page Arrow to see all Folders

Knowledge Check Where can you find the error message in File Manager? Download the Source File Hover over the Field highlighted in Red with your mouse Look for the X or E in each column and look at the Header of that column Knowledge Check

Dates must be in the format Where can you find the error message in File Manager? Download the Source File Hover over the Field highlighted in Red with your mouse Look for the X or E in each column and look at the Header of that column Dates must be in the format YYYY-MM-DD, such as 2017-06-30 Knowledge Check

Does a status of “Complete” in Batch Manager indicate all records were successfully uploaded? Yes No Only if the Data Status also has a Green Check Knowledge Check

Knowledge Check Only if the Data Status also has a Green Check Does a status of “Complete” in Batch Manager indicate all records were successfully uploaded? Yes No Only if the Data Status also has a Green Check Successful Knowledge Check Errors

Where can you find the ETL error message in Batch Manager (after files successfully passed DQE)? Hover over the Field highlighted in Red Look for the X or E in each column and look at the Header of that column View the error.zip file Knowledge Check

Where can you find the ETL error message in Batch Manager (after files successfully passed DQE)? Hover over the Field highlighted in Red Look for the X or E in each column and look at the Header of that column View the error.zip file Knowledge Check ERROR_VALID_VALUES vs ALERT_VALID_VALUES

For Assistance, contact PIMS Support at 800-661-2423