1. TSDS - Texas Student Data System PEIMS
Simple Solution. Brighter Futures.
TSDS - Texas Student Data System PEIMS
July 26, 2016
Texas Education Agency

2. Agenda Technical Updates Lessons Learned Partial file loading
Batch loading
ODS Deletions
Enhancements
Reports Bundling
Notifications
Release Schedule

3. Technical Updates ODS Statistics: Summer Collection:
 1,848,748,490 records processed through ODS for SUMR collection
Summer Collection:
68090 batches processed for the Summer Collections (containing 1.8 billion records)
70% of total jobs took less than 1 hour to process
14% of total jobs took longer than 2 hours to process
1% of the total jobs took longer than 8 hours to process
 Top 5 processing days for batches:
08-JUN-16 – total records uploaded 54,494,339 | 3411 batches processed | 2% processing longer than 2 hours
09-JUN-16 – total records uploaded 60,131,599 | 3919 batches processed | 4% processing longer than 2 hours
13-JUN-16 – total records uploaded 68,719,487 | 3967 batches processed | 10% processing longer than 2 hours
14-JUN-16 – total records uploaded 58,409,738 | 3766 batches processed | 4% processing longer than 2 hours
15-JUN-16 – total records uploaded 58,426,694 | 3399 batches processed | 20% processing longer than 2 hours
Database change made on 06/21/2016 to increase CPUs and optimize the processing of deletes:
resulted in a 13% increase in number of records processed between 06/21 and 06/22
resulted in a 19% increase in number of records processed on 6/13 compared to 6/22

4. Technical Updates, cont.
 ODS Planned Technical Work:
Working with eScholar to identify if system is processing at full capacity and processing queued jobs as fast and efficiently as possible
Determine why batches queued up, but not being processed
Add two (2) additional ETL servers
PEIMS Planned Technical Work:
Reviewing stored procedures for validations to identify performance improvement areas
Reviewing long running SQL each day to determine improvement areas
Determine why promotion jobs queued up, but not being processed
Review rules framework and implement changes where possible
Database Planned Work:
Increasing from two nodes to four nodes
Schedule weekly online maintenance and determine plans for monthly offline maintenance to ensure table space optimized
Review and implement recommendations from Oracle expert
Complete Oracle patching
Increasing temp space in PROD from 64g to 128g
Reviewing indexes to determine if indexing changes can help improve performance

5. Technical Updates, cont.
Reports Statistics:
FALL
Number of Reports generated in First Submission, previous framework: 135,884
Average execution time in seconds:  662 s
SUMMER
Number of Reports generated in First Submission, new & previous framework: 99,646
Average execution time in seconds: 39 s
Note: The Fall reports generated are much higher due to the number of fails/retries and also due to rollup.
Modifications made:
Added more report servers
Updated reports framework and technology

6. Lessons Learned Partial XML file loading in ODS Batch Loading
TEDS Field validation rules
Specific data changes
Ex: Discipline data
Batch Loading
One batch versus Multiple batches .

7. Lessons Learned, cont. Campus-level Validations ODS Deletions
LEAs running validations for each campus and the LEA at the same time. This takes significant processing time.
ODS Deletions
Student Category
Staff Category
Roll-up Reports
TEA is no longer running roll-up reports as LEAs can run reports as needed. All reports run in an Accepted collection will have Accepted in the title as well as the date Accepted in the footer.

8. Enhancements Reports Bundling Define, Edit, Delete, Schedule
Configure Parameters
Multiple Instances of one report
Bundle to zip files (optional)
Notifications of completion of bundle
Separate from regular reports
Data Completer Role – privilege “PEIMS Bundle Reports”
“PEIMS Generate Reports” privilege will allow user to generate the bundle
Limit number of reports an LEA can include
Set by TEA
LEA and Campus level
Auto-Generate after Validations
Fatal free only
Get a copy of requirements for assistance

9. Enhancements, cont. Notifications
ESC/LEA configurations have precedence over TEA configurations
Too many notifications at hierarchy level
Ex: Executive Directors
Get a copy of requirements for assistance

10. Release Schedule Unique ID: ODS: PEIMS Application:
August 4 – 7, 2016 version deployed to production.
Please note UID will be down during business hours.
ODS:
August 1, TEDS Final
October 3, TEDS Addendum
PEIMS Application:
September 1, TEDS Final
Promotions, Validations & Reports for new data elements will be available.
Validations and Reports (including Addendum changes) will be deployed as follows:
October 3, 2016: Fall
November 18, 2016: Midyear
January 27, 2017: Summer and Extended Year

11. Release Schedule, cont. studentGPS®Dashboards: ECDS:11
studentGPS®Dashboards:
August 1, 2016: Data can be loaded into the TSDS collection
Dashboards are available upon successful LEA onboarding
September 23, 2016: Historical current year metrics will be available
Drill down metrics and bar graphs for Attendance, Discipline and Grades
ECDS:
August 1, 2016: Data can be loaded into the ODS
January 27, 2017: All ECDS data can be promoted and validated