Managing Data & Information with the SECN Decision Support System GWS Meeting New Orleans, LA March 14-18, 2011

Outline Background and a brief history Background and a brief history Planning Process Planning Process Transition: MS Access to SQL Server Transition: MS Access to SQL Server SECN Decision Support System SECN Decision Support System Next Steps Next Steps

A Brief History Information Needs Assessment (INA) Information Needs Assessment (INA) Standard IT approach to systems planning Standard IT approach to systems planning Conceptual Modeling Develop Protocols Conceptual Modeling System Design Assess Needs Implement Conceptual Modeling Develop Protocols Assess Needs Conceptual Modeling Develop Protocols Implement Assess Needs Conceptual Modeling Develop Protocols

Outcomes of the Needs Assessment Preliminary protocol development summary documents Preliminary protocol development summary documents Protocol reporting (output) requirements and the necessary inputs to meet those requirements Protocol reporting (output) requirements and the necessary inputs to meet those requirements Preliminary conceptual object model Preliminary conceptual object model The foundation for database development The foundation for database development Describes required database components and the relationship between those components Describes required database components and the relationship between those components

Conceptual Object Model

Take home message To meet the needs of our program and the expectations of end-users, our data management system should be an integrated, enterprise level solution that takes advantage of the following: To meet the needs of our program and the expectations of end-users, our data management system should be an integrated, enterprise level solution that takes advantage of the following: Data integration across protocols Data integration across protocols Better performance, especially for larger datasets Better performance, especially for larger datasets Easier access across multiple locations Easier access across multiple locations Minimal data redundancy Minimal data redundancy Robust, online access to the centralized system Robust, online access to the centralized system Greater reporting capabilities Greater reporting capabilities

Status: Initial database development efforts in MS Access for our fixed-station water quality monitoring protocol Initial database development efforts in MS Access for our fixed-station water quality monitoring protocol Due to need for rapid database development Due to need for rapid database development Lack of approval from IT to use SQL Server Lack of approval from IT to use SQL Server Current staff not skilled in SQL server Current staff not skilled in SQL server

Fixed-station water quality monitoring Data loggers collect data every 30 minutes, year round Data loggers collect data every 30 minutes, year round >17,500 records, for 8 parameters, per station, per year >17,500 records, for 8 parameters, per station, per year By late 2005 there were 5 stations operating By late 2005 there were 5 stations operating Currently there are 17 stations in operation Currently there are 17 stations in operation Discrete nutrient data also collected monthly and quarterly Discrete nutrient data also collected monthly and quarterly Requires equipment inventory and maintenance records Requires equipment inventory and maintenance records Requires extensive pre- and post-deployment calibration Requires extensive pre- and post-deployment calibration Requires extensive QA/QC procedures using ancillary weather data and manual examination of the data Requires extensive QA/QC procedures using ancillary weather data and manual examination of the data

Database Problems for this Protocol Issues with Implementing this protocol database in Access: Issues with Implementing this protocol database in Access: Multiple users required multiple database copies – leading to problems with database replication Multiple users required multiple database copies – leading to problems with database replication QA/QC procedures difficult due to amount of data and performance issues QA/QC procedures difficult due to amount of data and performance issues Other Issues Other Issues Unable to centralize due to poor performance Unable to centralize due to poor performance Database size limits rapidly approaching Database size limits rapidly approaching Inadequate reporting capabilities Inadequate reporting capabilities

Lessons Learned Involve local IT staff early in the process and throughout the process Involve local IT staff early in the process and throughout the process Make sure your staff have the skills necessary to support whatever database and applications that you build. Make sure your staff have the skills necessary to support whatever database and applications that you build. Don’t start development until your protocol data collection procedures have been stabilized or finalized Don’t start development until your protocol data collection procedures have been stabilized or finalized

Skipping ahead to today During 2008, SECN transitioned from MS Access to SQL Server for existing databases During 2008, SECN transitioned from MS Access to SQL Server for existing databases Almost all new development takes place in SQL Server Almost all new development takes place in SQL Server Development begins when the protocol is stable Development begins when the protocol is stable Conceptual Modeling System Design Assess Needs Implement

SECN SharePoint Site Windows Server 2003 Windows Server 2003 SQL Server 2008, enterprise edition SQL Server 2008, enterprise edition SQL Server Reporting Services SQL Server Reporting Services Microsoft Office SharePoint Server (MOSS) 2007, enterprise edition Microsoft Office SharePoint Server (MOSS) 2007, enterprise edition Internet Information Services (IIS) 6.0 Internet Information Services (IIS) 6.0.NET framework 2.0 and 3.0.NET framework 2.0 and 3.0 Dundas Data Visualization for Reporting Services Dundas Data Visualization for Reporting Services Bing Maps Bing Maps Development tools: Microsoft InfoPath, Visual Studio Development tools: Microsoft InfoPath, Visual Studio

Functionality Overview Data Entry Data Entry Enter field data collection information Enter field data collection information Update database lookup tables Update database lookup tables Keep an inventory of equipment and repairs Keep an inventory of equipment and repairs Data QA/QC Data QA/QC Save an electronic copy of data sheet – as entered Save an electronic copy of data sheet – as entered Save an electronic copy of data sheet – as certified Save an electronic copy of data sheet – as certified Certify data entry Certify data entry For data logger data – flag as suspect, delete For data logger data – flag as suspect, delete Data Analysis and Reporting Data Analysis and Reporting Standardized reporting available on SharePoint Standardized reporting available on SharePoint Map viewer application Map viewer application

SECN SharePoint Site

Form is in MS InfoPath Form is in MS InfoPath Matches data sheet Matches data sheet * indicates required * indicates required Built in QA/QC Built in QA/QC Data submitted directly to the database as provisional data Data submitted directly to the database as provisional data Form saved as a record of data entry Form saved as a record of data entry Data verification is a separate step Data verification is a separate step

Update lookup tables

Inventory Equipment

QA/QC – Certify Data

Data Analysis and Reporting

Next Steps Continuing development for protocols coming online Continuing development for protocols coming online Expanding our reporting capabilities and data access tools Expanding our reporting capabilities and data access tools Working with US-FWS I&M program to take advantage of our system whenever protocols allow Working with US-FWS I&M program to take advantage of our system whenever protocols allow Moving our mapping application to NPMap Moving our mapping application to NPMap

Questions?