TSDS - Texas Student Data System PEIMS Simple Solution. Brighter Futures. TSDS - Texas Student Data System PEIMS July 26, 2016 Texas Education Agency

Agenda Technical Updates Lessons Learned Partial file loading Batch loading ODS Deletions Enhancements Reports Bundling Notifications Release Schedule

Technical Updates ODS Statistics: Summer Collection:  1,848,748,490 records processed through ODS for SUMR collection Summer Collection: 68090 batches processed for the Summer Collections (containing 1.8 billion records) 70% of total jobs took less than 1 hour to process 14% of total jobs took longer than 2 hours to process 1% of the total jobs took longer than 8 hours to process  Top 5 processing days for batches: 08-JUN-16 – total records uploaded 54,494,339 | 3411 batches processed | 2% processing longer than 2 hours 09-JUN-16 – total records uploaded 60,131,599 | 3919 batches processed | 4% processing longer than 2 hours 13-JUN-16 – total records uploaded 68,719,487 | 3967 batches processed | 10% processing longer than 2 hours 14-JUN-16 – total records uploaded 58,409,738 | 3766 batches processed | 4% processing longer than 2 hours 15-JUN-16 – total records uploaded 58,426,694 | 3399 batches processed | 20% processing longer than 2 hours Database change made on 06/21/2016 to increase CPUs and optimize the processing of deletes: resulted in a 13% increase in number of records processed between 06/21 and 06/22 resulted in a 19% increase in number of records processed on 6/13 compared to 6/22

Technical Updates, cont.  ODS Planned Technical Work: Working with eScholar to identify if system is processing at full capacity and processing queued jobs as fast and efficiently as possible Determine why batches queued up, but not being processed Add two (2) additional ETL servers PEIMS Planned Technical Work: Reviewing stored procedures for validations to identify performance improvement areas Reviewing long running SQL each day to determine improvement areas Determine why promotion jobs queued up, but not being processed Review rules framework and implement changes where possible   Database Planned Work: Increasing from two nodes to four nodes Schedule weekly online maintenance and determine plans for monthly offline maintenance to ensure table space optimized Review and implement recommendations from Oracle expert Complete Oracle patching Increasing temp space in PROD from 64g to 128g Reviewing indexes to determine if indexing changes can help improve performance

Technical Updates, cont. Reports Statistics: FALL Number of Reports generated in First Submission, previous framework: 135,884 Average execution time in seconds:  662 s   SUMMER Number of Reports generated in First Submission, new & previous framework: 99,646 Average execution time in seconds: 39 s Note: The Fall reports generated are much higher due to the number of fails/retries and also due to rollup. Modifications made: Added more report servers Updated reports framework and technology

Lessons Learned Partial XML file loading in ODS Batch Loading TEDS Field validation rules Specific data changes Ex: Discipline data Batch Loading One batch versus Multiple batches .

Lessons Learned, cont. Campus-level Validations ODS Deletions LEAs running validations for each campus and the LEA at the same time. This takes significant processing time. ODS Deletions Student Category Staff Category Roll-up Reports TEA is no longer running roll-up reports as LEAs can run reports as needed. All reports run in an Accepted collection will have Accepted in the title as well as the date Accepted in the footer.

Enhancements Reports Bundling Define, Edit, Delete, Schedule Configure Parameters Multiple Instances of one report Bundle to zip files (optional) Notifications of completion of bundle Separate from regular reports Data Completer Role – privilege “PEIMS Bundle Reports” “PEIMS Generate Reports” privilege will allow user to generate the bundle Limit number of reports an LEA can include Set by TEA LEA and Campus level Auto-Generate after Validations Fatal free only Get a copy of requirements for assistance

Enhancements, cont. Notifications ESC/LEA configurations have precedence over TEA configurations Too many notifications at hierarchy level Ex: Executive Directors Get a copy of requirements for assistance

Release Schedule Unique ID: ODS: PEIMS Application: August 4 – 7, 2016 version 9.0.2 deployed to production. Please note UID will be down during business hours. ODS: August 1, 2016 2016-2017 TEDS Final October 3, 2016 2016-2017 TEDS Addendum PEIMS Application: September 1, 2016 2016-2017 TEDS Final Promotions, Validations & Reports for new data elements will be available. 2016-2017 Validations and Reports (including Addendum changes) will be deployed as follows: October 3, 2016: Fall November 18, 2016: Midyear January 27, 2017: Summer and Extended Year

Release Schedule, cont. studentGPS®Dashboards: ECDS: 11 studentGPS®Dashboards: August 1, 2016: Data can be loaded into the TSDS collection Dashboards are available upon successful LEA onboarding September 23, 2016: Historical current year metrics will be available Drill down metrics and bar graphs for Attendance, Discipline and Grades ECDS: August 1, 2016: Data can be loaded into the ODS January 27, 2017: All ECDS data can be promoted and validated