UNIVERSITY OF UTAH GREEN INFRASTRUCTURE MONITORING DATABASE CVEEN 7970 Hydroinformatics Semester Project Zachary Magdol, Jai Kanth Panthail, Pratibha Sapkota, Megan Walsh 1

Outline Introduction GI sites around the University GIRF(Green Infrastructure Research Facility) ODM schema for GIRF Database Data and Database Requirements Implementing MSQL Database Challenges of implementing the database for GIRF PYTHON for Data Analysis Calculations with Python Plotting in PYTHON DBPlot (Alternative Data Analysis Tool) Future Work 2

Green Infrastructure Monitoring Green infrastructure sites in and around U of U campus employ sensors to monitor various parameters for research and performance evaluation Problem: No framework for organizing data No method for compiling data No interface for visualizing/analyzing data 3

Green Infrastructure Monitoring Variables Soil moisture Soil temp Flow Depth Water quality Climatic data 4

Green Infrastructure Research Facility (GIRF) 5

GIRF 6

Data m Gravel 0.6 m Topsoil 4.5 m 2 m Impermeable Liner Transp iration Infiltration Drain Synthetic Storm Water Sensors monitor soil moisture and soil temperature at different depths and locations Battery powers sensors and data logger Data logger collects soil moisture, soil temperature and battery voltage observations every 30 minutes

Current data management plan No set technique for compiling, organizing, and sharing data from the sensors Data collected is downloaded as CSV files, and they are analyzed using Excel. 8 Problems:  Cumbersome process  Very difficult to find relationships within data  Analysis is limited using Excel Need a proper data management plan to make life easier for everyone!

Sample data: 9

10 Data storage User interface Data Analysis Data retrieval and input Main requirements: 1.Automated data entry and storage 2.Easy querying of data 3.Connections with analysis programs and code 4.Easy user interface 5.Suitable for future upgrades and modifications GIRF Data Management Plan User Interface (Hydrodesktop, Custom GUI or Webpage) Analysis tools (Python, R, Matlab, DB Plot etc) Database (MySQL, Microsoft SQL etc) Automated data input (Wireless links, Telemetry etc)

GIRF Database Framework Schema requirements: 1. Support for relational queries. 2. Built to be suitable for storing hydrological data. 3. Sufficient support for storing metadata 4. Background data like Variables and Units is already present in the database schema 5. Compatibility with CUAHSI tools like HydroDesktop and HydroServer Lite - Future opportunity! 6. Easy implementation in MySQL 11 We required a suitable database schema to suitably store the GIRF data…

Option 1: Custom database Schema 1 12 Developed by Meg Walsh

13 Option 1: Custom database Schema 2 Developed by Zach Magdol

14 Option 3: ODM schema- Selected option Source:

15 Challenge 1: The ODM schema was not originally designed to store data from bioretention plots. At the University of Utah, each plot (a site) has individual bioretentions (sub-sites).  Each bioretention plot was considered to be a site and was added separately to the ‘Sites’ table. Adding sites specific to GIRF Database implementation -> Using MySQL Command Line Client and TOAD ®

16 Adding variables specific to GIRF Challenge 2: Each bioretention plot (a ‘Site’ in the GIRF database) had sensors at multiple depths. For example, moisture sensors existed at depths of 10cm, 25cm, 30cm and 50cm.  Four new variables representing the moisture at individual depths had to be created. This was required to preserve the dissimilarity of data from each depth.

17 Sample data added using TOAD  Data for GIRF-Upland-1, GIRF-Control-2 and GIRF-Wetland-3  Variables consisted of UOUVolt, UOUTemp5, UOUMoist10 and UOUMoist25  Total of approximately values were added to the ‘datavalues’ table

Data Analysis 18 Python

MySQLdb Python interface Free! connects to the database with the connection command, or “cnxn”. 19

Python Matplotlib.pyplot 2D plotting library Free! Numpy enables complex scientific computing to be performed in python Free! 20 Source: Source:

Python Analysis & Results 21 Get Values Average Daily Temperature

Python Advantages Free Easy to use Easy to connect with MySQL databases Plotting capabilities: user-defigned plot preferences Disadvantages Indentations Must download/import appropriate modules Numerous lines of code 22

Alternative Data Analysis Tool DBPlot ( GUI to visualize and manipulate data Supports MySQL, MS SQL, SQLite 23 Query script Query results Query status

DBPlot Compute and plot daily average, maximum and minimum soil moisture at 10 cm depth in Site 1 select EXTRACT(day from localdatetime) AS theday, EXTRACT(year from localdatetime) AS theyear, EXTRACT(month from localdatetime) AS themonth, avg(datavalue) AS avg_per_cent, min(datavalue) AS min_per_cent, max(datavalue) AS max_per_cent from datavalues where siteid=1 and variableid=5 group by EXTRACT(day from localdatetime) order by theday 24

DBPlot Advantages: Easy to use Same language as database Rapidly query and visualize data in one tool Disadvantages: Poor plot quality and customization capabilities Limited analyses 25

Conclusion Successfully loaded GI data into ODM schema Data uploaded to MySQL using Toad Explored data analysis alternatives Python DBPlot R 26

Future Work Expand current database to include data from all GI sites in the University Automate data acquisition and analysis Customize user-friendly interface to allow data visualization Create a GI website, and connect to the database for data sharing 27