May 2013 Mass Report Generation Managing High Volume Reporting
Overview of Volume Output Requirements Common Problems Report Resolution Demonstration Overview of Volume Output Requirements Common Problems Report Resolution Demonstration Agenda
In 2012 the WorleyParsons Environmental Data Management Team processed: 6,500 lab EDDs, From 4 primary laboratories, accounting for 28,000 samples, and 1,436,000 results, Into 3 databases containing 3,000 facilities across Canada Overview of Volume
Common data table and graph outputs are displayed: Historically, year over year With guidelines (action levels) comparisons applied Grouped by analytes From scratch on a regular basis (i.e. quarterly) Output Requirements Table 7 of 8, Page 105 of 132 for 1 of 11 sites Graphing sheet 3 of 800
Staffing: It takes ~6 months for staff to get up to speed After 2.5 years they are burnt out Timing: We have several programs that require reporting to be in the clients hand within 24 hours of sample drop off Laboratory issues: Inaccurate or incomplete reporting Reporting what we want, not what we need Labs sign reports, not data System Issues: An x86 32 bit machine has a 2 GB memory limit Large data sets using a modified EQuIS report are time consuming, taking over 45 minutes to run Cross tab display uses 5mb/second Exporting to Excel 20mb/sec Common Problems Does anyone have an analyte group that looks like this? One analyte, multiple methods.
In EQuIS 5.6 EarthSoft introduced a new Beta analytical report Allows for a new approach to reporting, using dynamic SQL Combined with a modified lab reporting methodology WorleyParsons has been able to reduce reporting times significantly Core features include: Significant time and memory reduction −47,000 record report (pulled in 23 seconds) −cross-tabbed to 5,785 (37 seconds) −with guidelines applied (41 seconds) −Similar data pulled in standard EQuIS report without guideline comparison and no-cross tab (90 seconds) Simplified analytical grouping Auto Cross-Tab functionality without significant memory loss Direct Guideline (Action Level) comparison Location Specific Guideline Comparison Direct Web GIS Interfacing Report Resolution
Demonstration (Standard Analytical Report) Pick Selection
Analytical Report Applied to Web GIS Analytical Report Pick Interface Web GIS Report Results
Thank you.