Batch Uploads In the ODM Using Worldwater.byu.edu A Yellowstone Trophic State Data Preservation Project Ben Thompson Jeff Sadler Derek Whitman

Introduction Currently uploads are accomplished through two general methods. The first one is to upload a csv file with a specific format for any single site with a single source for one single type of variable with many values for different times. The second it to upload data using forms provided within the site. In both of these methods the site, source, and variable type must already be created within the ODM for the data entries to work. The goal of this project was to improve the way data is uploaded to the ODM through HydroServer Interactive by creating a csv file upload routine which will process multiple points in multiple sites and multiple data types as well as create new sites when needed. For the sake of simplicity, the project scope was limited to a single database within the ODM using a single source.

Background and Gaps Single, Multiple CSV (only one site, variable, and source) Many people have data in spreadsheets

Gathering Data Dr. Woodruf Miller from BYU has been gathering data from surveys of the lakes in Yellowstone National Park and Teton National Park for the last 10 years or so. This data includes phosphorus content, chlorophyl content, and sechi depths of lakes. These data have been used to calculate trophic state indexies of the lakes using different trophic state index calculation methods. The flow rates of inlets and outlets of several of the lakes are also included in survey data.

Formatting and Organizing the Data

Formatting and Organizing the Data Total Phosphorus (ppb) Total Phosphorus (mg/L) Flow (cfs) Hydraulic Residence Time (yrs) Phosphorus Retention Coef. Primary Production Carlson TSI Burns TLI Naumann TI Site Year Month Location Lat Lon Secchi Depth Chlorophyll-a (mg/m^3)

Creating the Batch Upload Three Main Parts HTML PHP Python

HTML http://worldwater.byu.edu/millerDataTest/index.html

HTML

PHP Successfully upload the file to a known location on the server Two Main Objectives Successfully upload the file to a known location on the server Run the Python script

PHP Uploading the File

Running the Python Script PHP Running the Python Script

Function to get Variable ID Python Three Components Main script Function to get Site ID Function to get Variable ID

Overview HTML PHP Upload/Save Run Python Python Ajax check Send printed messages PHP 2 Create Connection Input Data Update Series Catalog Get Variable ID Go to new data row Go to new data value Get Site ID

Results 116 sites 3700+ data values New consistent format for recording survey data Massive time savings

Results

Conclusion Usefulness of tools like Python Data can be easily shared (CUAHSI HIS) Future batch uploads Improvements- sources and variables