1. Disease Monitoring with SQL Server BI
SQL Server Business Intelligence for disease surveillance and monitoring
Matt F. Smith
SQL Server Data Engineering
Presentation for SQL Saturday #596, Denver CO USA

2. > whoami
Enjoy working with the MSFT SQL stack, TSQL code & infrastructure
My recent learning has been around Cloudera, HDFS, Sqoop, Flume and Spark…I am very excited for SQL Server on Linux
I write code every day
Interactive presentation. If something is not clear, speak up. Let’s make this a dialogue.

3. Today’s Agenda A BI Solution Overview Big Words & Useful Terminology.
A SQL Server BI Solution for Disease Surveillance and Monitoring.
Other interesting things…
Interactive presentation. If something is not clear, speak up. Let’s make this a dialogue.

4. Relevant terminology
Epidemiology: Study and analysis of the patterns, causes, and effects of health and disease conditions in defined populations
MMWR: Morbidity and Mortality Weekly Report (MMWR)
Diagnosis of Tuberculosis in Three Zoo Elephants and a Human Contact — Oregon, 2013
You can subscribe on the CDC web site!
Incident Proportion (important metric!) : [# of new cases during a specific time period] / [Population] (varies by neighborhood)
5-week rolling window: (current week +- 2 weeks), last 5 years. Used to trend and compare current rate (now) to historical rate.
Standard Deviation: sqrt of variance (T-SQL STDEV)
Alert: >=1sd and <2sd
Outbreak: >=2sd

5. ETL Architecture Systems Overview
Software: Microsoft SQL Server Stack (SQL 2016, SSIS, SSRS, PowerBI Desktop)
Infra: SQL Server running on VmWare, Dell Servers
(Please see Dell/SQL Server Reference Architectures)
Systems Overview
Flat files and MS Access databases dropped on network share from partners via sFTP
SQL Agent Jobs execute SSIS packages to pick files and load into raw data staging area (minimal transformations in this stage)
Composite primary keys are created from incoming data, hashes created for keys and row data, custom CDC process merges raw data into secondary data staging area
Data is loaded into data warehouse – Normalized EDW containing 30+ source systems, schema based on HL7 data model
Data is provisioned into a data mart for reporting and analysis twice daily
SSRS points to data mart, mart contains some SQL Server views for access by named power users who query from SSMS, Microsoft Excel and Microsoft Power BI

6. Reporting & BI Architecture
SQL Server Reporting Services, Microsoft Excel, Power BI
SQL Server Reporting Services
SSRS reports are useful for ~80% of customer base
Dashboards: Anything that has alerted or outbreak over the past 5 weeks (let’s keep an eye on it)
Relevant information from incoming phone calls: Disease incidents
Team Metrics: Incoming Disease Count, Count for Investigation, Count Closed, etc.
SSRS Mapping components used extensively – Data points geo-coded for neighborhood mapping, web services for accurate geo-coding
SQL Server Views reference data for Power BI users
Views present data from a data mart
Customers simply pull data into excel and pivot

7. Project Team
Lean team, 1- PM/Business Analyst, 2 – Developers/QA/Analyst
Experienced team has breadth and depth to deliver the project (It’s about quality, not quantity!)
Methodology: Scrum-fall – Some requirements pre-defined, Agile used to evolve/work through these and other undefined requirements
PM does BA work in addition to project management
Developers do BA work, code, write test cases, refine user stories, meet and work with end-users
Product owners, developers, PM, major stakeholders attend daily scrum
Stakeholders accept work, perform their own QA using SSMS & excel, are most familiar with the data and the reporting requirements (internal as well as external reporting – State of CO)

8. Challenges Not being an Epidemiologist – knowledge gap
Solution: Document process, learn/engage/understand stakeholder operations, partner with customer and share information. Grow the relationship.
Schedule and availability of resources to assist with QA
Test driven development ensures tests are written before development starts. T-SQL tests help make QA meetings move more quickly

9. Stakeholders
Tracking, Investigating alerts and outbreaks of infectious disease
Epidemiology Team: 12 persons: Epidemiologists, Nurses, Managers, Directors, other team members
Incoming cases tracked, filtered by disease type, assigned to individuals for follow- up (interviews)
Outbreaks or major issues (Measles, Mumps) may require participation from entire team until investigation is complete
State of Colorado - reporting

10. …and yes, zombies are real.
Zombie Ants: adaptive parasite manipulation
Fungus Makes Zombie Ants Do All the Work
A tropical fungus has adapted to infect ants and force them to chomp, with surprising specificity, into perfectly located leaves before killing them and taking over their bodies
By Katherine Harmon on July 31, 2009