R: Packages and Data Retrieval

Slides:



Advertisements
Similar presentations
GEOSS Water Services for Data and Maps Community Recommendations David K. Arctur, Project Lead, GEOSS AIP-6 Water SBA University.
Advertisements

Some notes on CyberGIS in hydrology Ilya Zaslavsky Spatial Information Systems Lab San Diego Supercomputer Center UCSD TeraGrid CyberGIS Workshop, February.
Harish Sangireddy The University of Texas at Austin.
HydroServer A Platform for Publishing Space- Time Hydrologic Datasets Support EAR CUAHSI HIS Sharing hydrologic data Jeffery.
Linking HIS and GIS How to support the objective, transparent and robust calculation and publication of SWSI? Jeffery S. Horsburgh CUAHSI HIS Sharing hydrologic.
This work is funded by National Science Foundation Grant EAR Accessing and Sharing Data Using the CUAHSI Hydrologic Information System CUAHSI HIS.
Terminology and Controlled Vocabulary Efforts at the U.S. Environmental Protection Agency Richard Huffine Federal Manager, EPA National Library Network.
CUAHSI HIS Data Services Project David R. Maidment Director, Center for Research in Water Resources University of Texas at Austin (HIS Project Leader)
Crossing the Digital Divide
Time Series Analyst An Internet Based Application for Viewing and Analyzing Environmental Time Series Jeffery S. Horsburgh Utah State University David.
Development of a Community Hydrologic Information System Jeffery S. Horsburgh Utah State University David G. Tarboton Utah State University.
Integrating Historical and Realtime Monitoring Data into an Internet Based Watershed Information System for the Bear River Basin Jeff Horsburgh David Stevens,
Introducing the CUAHSI Hydrologic Information System Desktop Application (HydroDesktop) and Open Development Community Jiří Kadlec, Daniel Ames, Teva Velupillai.
An End-to-End System for Publishing Environmental Observations Data Jeffery S. Horsburgh David K. Stevens, David G. Tarboton, Nancy O. Mesner, Amber Spackman.
Colorado’s Surface Water Monitoring System Colorado Division of Water Resources.
U.S. Department of the Interior U.S. Geological Survey NWIS Web Services Snapshot for ArcGIS Sally Holl and David Maltby Based on work by David McCulloch.
Ven Te Chow Award Lecture David R. Maidment ASCE EWRI World Conference Palm Springs, 24 May 2011.
CE 397 Midterm Slides Review Slides for the Midterm Exam David Maidment Spring 2015.
About CUAHSI The Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) is an organization representing 120+ universities.
Ocean Sciences What is CUAHSI? CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc Formed in 2001 as a legal entity Program.
Exercises: Organizing, Loading, and Managing Point Observations Using HydroServer Support EAR CUAHSI HIS Sharing hydrologic data
Water Web Services David R. Maidment Center for Research in Water Resources University of Texas at Austin Open Waters Symposium Delft, the Netherlands.
Publishing Observations Data: from ODM to HIS Central.
Water and Catchment Data Services David R. Maidment Center for Research in Water Resources University of Texas at Austin River Science Symposium Swansea,
U.S. Department of the Interior U.S. Geological Survey USGS Water Data Exchange Services USGS Office of Water Information June 2009 Nate Booth, Dave Briar.
_______________________________________________________________CMAQ Libraries and Utilities ___________________________________________________Community.
Water Web Services. Connecting a Catalog with Users and Servers Server User Catalog Data Services HydroServerHydroDesktop HIS Central Data Services WISKI.
Bringing Water Data Together David R. Maidment Center for Research in Water Resources University of Texas at Austin Texas Water Summit San Antonio Tx,
Critical Zone Observatory Data Discovery Each CZO maintains its own data management system(s) using the data formats it prefers The three CZO’s have a.
Exporting WaterML from the Earth System Modeling Framework Xinqi Wang Louisiana State University NCAR SIParCS Program August 4, 2009.
September 2012 Developed by Agricultural and Biological Engineering Department at Purdue University and Department of Regional Infrastructures Engineering.
Sharing SRP Water Sample Data Using CUAHSI HIS Infrastructure Ilya Zaslavsky, Thomas Whitenack, Keith Pezzoli, Hiram Sarabia University of California at.
The CUAHSI Observations Data Model Jeff Horsburgh David Maidment, David Tarboton, Ilya Zaslavsky, Michael Piasecki, Jon Goodall, David Valentine,
CUAHSI HIS: Science Challenges Linking small integrated research sites (
GEO 5159 GIS Applications in Environmental Systems Seminar 3: Question, Models, Data, Download, Learning.
A GIS approach to understanding groundwater – surface water interactions in the Logan River and Red Butte Creek, Utah Trinity Stout CEE 6440.
U.S. Department of the Interior U.S. Geological Survey Water Data Sharing Using OGC Standards and the Internet David Blodgett USGS Center for Integrated.
Google Code Libraries Dima Ionut Daniel. Contents What is Google Code? LDAPBeans Object-ldap-mapping Ldap-ODM Bug4j jOOR Rapa jongo Conclusion Bibliography.
Lecture 11 Introduction to R and Accessing USGS Data from Web Services Jeffery S. Horsburgh Hydroinformatics Fall 2013 This work was funded by National.
Data Management: Data Processing Types of Data Processing at USGS There are several ways to classify Data Processing activities at USGS, and here are some.
Hydroinformatics Lecture 15: HydroServer and HydroServer Lite The CUAHSI HIS is Supported by NSF Grant# EAR CUAHSI HIS Sharing hydrologic data.
UNIVERSITY OF UTAH GREEN INFRASTRUCTURE MONITORING DATABASE CVEEN 7970 Hydroinformatics Semester Project Zachary Magdol, Jai Kanth Panthail, Pratibha Sapkota,
Using Python to Retrieve Data from the CUAHSI HIS Web Services Jeffery S. Horsburgh Hydroinformatics Fall 2015 This work was funded by National Science.
Statistical Exploratory Analysis with “EnQuireR” 1.Introduction 2.Installation 3.How to 4.Report.
Desktop Database and Climate Analysis Steven Burian and Erfan Goharian Hydroinformatics Fall 2013.
Using Python to Retrieve Data from the CUAHSI HIS Web Services Jon Goodall Hydroinformatics Fall 2014 This work was funded by National Science Foundation.
Developing a community hydrologic information system David G Tarboton David R. Maidment (PI) Ilya Zaslavsky Michael Piasecki Jon Goodall
The CUAHSI Hydrologic Information System Spatial Data Publication Platform David Tarboton, Jeff Horsburgh, David Maidment, Dan Ames, Jon Goodall, Richard.
Red Butte Creek Storm Drain Sensor Network
Using GIS in Creating an End-to-End System for Publishing Environmental Observations Data Jeffery S. Horsburgh David G. Tarboton, David R. Maidment, Ilya.
Sharing Hydrologic Data with the CUAHSI* Hydrologic Information System
R: Working with Databases
Contents Module 6: E-journal, E-books and Internet Resources
R: Packages and Data Retrieval
INTRODUCTION TO GEOGRAPHICAL INFORMATION SYSTEM
Introduction to Visual Basic 2008 Programming
Getting Started with R.
Lecture 8 Database Implementation
PYTHON: AN INTRODUCTION
CUAHSI HIS Sharing hydrologic data
DATA MINING Python.
HydroDesktop: A Key Component of the CUAHSI/CZO HIS for Hydrologic Data Discovery, Visualization, and Analysis Daniel P. Ames, Ph.D. P.E. Idaho State University.
Community Tools and Support for Data Driven Education
SDMX: Enabling World Bank to automate data ingestion
David Tarboton, Dan Ames, Jeffery S. Horsburgh, Jon Goodall
A (prototype) Shiny app for QCing continuous stream sensor data
Comments, Prototypes, Headers & Multiple Source Files
Analytics: Its More than Just Modeling
ESRI User Conference Water Resources User Group, July 15, 2010
Fundamental Science Practices (FSP) of the U.S. Geological Survey
Presentation transcript:

R: Packages and Data Retrieval Hydroinformatics – Fall 2015

Learning Objectives Describe the difference between packages and libraries in R Install and load packages Use documentation and other resources to learn how to use unfamiliar packages Use data retrieval packages and web services to obtain hydrologic data

R Packages and Libraries Package: R functions, data, and compiled code in a well- defined format. Library: The directory where packages are stored.  Implement many common data analysis and statistical procedures Provide excellent graphics functionality Serve as a starting point for many data analysis tasks A huge community of R developers exist – it’s likely that there’s an R package for many of the tasks you commonly do

USGS Data Retrieval Package for R Collection of functions to help retrieve hydrologic and water quality data using web services: U.S. Geological Survey (USGS) using National Water Information System (NWIS) tools U.S. Environmental Protection Agency (EPA) Data access is through web services https://github.com/USGS-R/dataRetrieval

Installing and loading Packages library() #displays available packages Tools>Install Packages or install.packages(“package_name”) library(package_name) #loads package

Demo – dataRetrieval We want to automate the retrieval and import of data from a specific site into R using the dataRetrieval package. But how? https://github.com/USGS-R/dataRetrieval Use the source, Luke!

Demo – dataRetrieval for Gage Height Try: Retrieve USGS gage height data for any site in Utah for May 2014. What function should you use? This package is well documented and its examples are very useful for modifying/tailoring to specific needs. For example, this code uses “service” (look at the examples and see that it is “iv” for one readNWISdata() and it is “site” for another. What do these terms mean, and where can you find out about them?

Demo – dataRetrieval for Gage Height How do you find the parameter code for gage height? Explore the github documentation: If you know a package can help you, but you don’t know how to use the functions, you may have to explore! Find examples that you can modify: Search the comments in the function code:

Demo – dataRetrieval for Gage Height Find site code: #Then modify the other example code with your site ID and parameter code instGage <- readNWISdata(sites="10171000", service="iv",parameterCd="00065", startDate='2014-05-01T00:00Z',endDate='2014- 05-31T00:00Z') UT 00065 From the examples in the readNWISdata function

WaterML Package for R WaterML is a standard information model for the representation of water observations data, with the intent of allowing the exchange of such data sets across information systems.

WaterML Package for R Developed as part of a semester project from last year, Jiri and Bryn from BYU

WaterML Package for R retrieve and analyze data from HydroServers of multiple organizations that are listed in the CUAHSI Water Data Center catalog  great example of semester project

Demo - WaterML Try downloading two variables from Red Butte Creek, Logan River, or Provo River GAMUT sites and fit a regression model between the two datasets https://www.cuahsi.org/Files/Posts/documents/WDC_Hand out.pdf