Director’s Report Unidata Users Committee Meeting 13 October 2005 Boulder, CO Mohan Ramamurthy Unidata Program Center UCAR Office of Programs Boulder,

Slides:



Advertisements
Similar presentations
LEAD Portal: a TeraGrid Gateway and Application Service Architecture Marcus Christie and Suresh Marru Indiana University LEAD Project (
Advertisements

1 NASA CEOP Status & Demo CEOS WGISS-25 Sanya, China February 27, 2008 Yonsook Enloe.
White House New Media & Open Source Software Macon Phillips White House New Media.
Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.
PTAGIS Program Review and Roadmap Focused on Portland Team for the 2014 PIT Tag Steering Committee Annual Meeting.
WSUS Presented by: Nada Abdullah Ahmed.
The LEAD Effort at Unidata The Unidata Seminar will start at 1:30 PM MST.
A New Generation of Data Services for Earth System Science Education and Research: Unidata’s Plans and Directions AGU Fall Meeting San Francisco, CA 6.
Director’s Report Unidata Policy Committee Meeting 5 February 2004 Mohan Ramamurthy Unidata Program Center UCAR Office of Programs Boulder, CO.
Integrating Historical and Realtime Monitoring Data into an Internet Based Watershed Information System for the Bear River Basin Jeff Horsburgh David Stevens,
GEMPAK-AWIPS II Discussion Unidata Policy Committee Meeting 12 May 2008.
Africa Information Highway and SDMX implementation in Africa Beejaye Kokil Economic & Social Statistics Division African Development Bank
TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.
Unidata TDS Workshop THREDDS Data Server Overview October 2014.
Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process.
Jordan G. Powers Mesoscale and Microscale Meteorology Division NCAR Earth System Laboratory National Center for Atmospheric Research Space Weather Workshop.
October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.
CCSM Portal/ESG/ESGC Integration (a PY5 GIG project) Lan Zhao, Carol X. Song Rosen Center for Advanced Computing Purdue University With contributions by:
Near East Rural & Agricultural Knowledge and Information Network - NERAKIN Food and Agriculture Organization of the United Nations Near East and North.
Quick Unidata Overview NetCDF Workshop 25 October 2012 Russ Rew.
Accessing the Amazon Elastic Compute Cloud (EC2) Angadh Singh Jerome Braun.
RIDGE Version II (Radar Integrated Display with Geospatial Elements) NOAA Systems of Systems Workshop – October 6-7, 2009 – RIDGE Team  RIDGE Version.
Unidata’s TDS Workshop TDS Overview – Part II October 2012.
Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.
Web Advisory Group (WAG) Implementation Plan ITC 10/19/04 Markus Stobbs.
Director’s Status Report May 19, 2003 Unidata Policy Committee Meeting National Science Foundation Arlington, VA Director’s Report Mohan K. Ramamurthy.
U.S. Department of the Interior U.S. Geological Survey Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center.
Providing data services, tools and cyberinfrastructure leadership Unidata Policy Committee May 2011 Organizational Collaboration, Participation,
Bonanza Creek LTER Information Management Information Management at BNZ Information Management Data & Metadata Website & Communication Education.
N-Wave Stakeholder Users Conference Wednesday, May 11, Marine St, Rm 123 Boulder, CO Linda Miller and Mike Schmidt Unidata Program Center (UPC)-Boulder,
The IDV: Unidata’s Integrated Data Viewer Mike Voss Department of Meteorology SJSU – Oct 11, 2006.
Director’s Report Unidata Policy Committee Meeting 24 May 2004 Mohan Ramamurthy Unidata Program Center UCAR Office of Programs Boulder, CO.
Accomplishments and Remaining Challenges: THREDDS Data Server and Common Data Model Ethan Davis Unidata Policy Committee Meeting May 2011.
The Arctic Observing Network (AON) Cooperative Arctic Data and Information Service (CADIS) Florence Fetterer,
THORPEX Interactive Grand Global Ensemble (TIGGE) China Meteorological Administration TIGGE-WG meeting, Boulder, June Progress on TIGGE Archive Center.
Unidata TDS Workshop THREDDS Data Server Overview
1 NASA CEOP Status & Demo CEOS WGISS-24 Oberpfaffenhofen, Germany October 15, 2007 Yonsook Enloe.
Quick Unidata Overview NetCDF Workshop 2 August 2009 Russ Rew Data Services Group.
EOSDIS Status 10/16/2008 Dan Marinelli, Science Systems Development Office.
David R. Maidment Unidata Program Center, Boulder CO 6 Feb 2004
THREDDS Catalogs Ethan Davis UCAR/Unidata NASA ESDSWG Standards Process Group meeting, 17 July 2007.
Ligia Bernardet, S. Bao, C. Harrop, D. Stark, T. Brown, and L. Carson Technology Transfer in Tropical Cyclone Numerical Modeling – The Role of the DTC.
Unidata’s TDS Workshop TDS Overview – Part I July 2011.
Director’s Report Policy Committee Meeting 9 October 2003 Mohan Ramamurthy Unidata Program Center UCAR Office of Programs Boulder, CO.
LEAD – WRF How to package for the Community? Tom Baltzer.
Near Real-Time Verification At The Forecast Systems Laboratory: An Operational Perspective Michael P. Kay (CIRES/FSL/NOAA) Jennifer L. Mahoney (FSL/NOAA)
MADIS Airlines for America Briefing Meteorological Assimilated Data Ingest System (MADIS) FPAW Briefing Steve Pritchett NWS Aircraft Based Observations.
The Unified Access Framework for Gridded Data … the 1 st year focus of NOAA’s Global Earth Observation Integrated Data Environment (GEO-IDE) Steve Hankin,
An Update on COLA’s Software Development Jennifer M. Adams and Brian Doty.
Presented to: By: Date: Federal Aviation Administration NextGen Network Enabled Weather Capability Evaluations Demonstration and Prototyping Information.
29 March 2004 Steven Worley, NSF/NCAR/SCD 1 Research Data Stewardship and Access Steven Worley, CISL/SCD Cyberinfrastructure meeting with Priscilla Nelson.
Unidata Users Committee Tom Whittaker Chair Summary of April Meeting Boulder, CO.
IDV Perspective: Climatology of the 2005 Hurricane Season Shelley O. Holmberg University of North Carolina at Charlotte, Charlotte, NC Brian J. Etherton.
Presentation to Unidata User's Committee September 2006 IDV – Past, Present, Future Don Murray and Jeff McWhirter.
UCAR and NCAR at is UCAR and NCAR’s 50 th Anniversary HAO’s 70 th Anniversary COMET’s 20 th Anniversary Unidata’s 25 th Anniversary
May 2010 GGIM, New York City The National System for Coordination of Territorial Information SNIT NSDI of Chile.
The Earth Information Exchange. Portal Structure Portal Functions/Capabilities Portal Content ESIP Portal and Geospatial One-Stop ESIP Portal and NOAA.
1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)
5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.
Interoperability Day Introduction Standards-based Web Services Interfaces to Existing Atmospheric/Oceanographic Data Systems Ben Domenico Unidata Program.
Serving Iowa Mesonet data with U of Minnesota’s MapServer Daryl Herzmann Iowa Environmental Mesonet 31 Jul 2002.
Data Browsing/Mining/Metadata
AWRA – Open Water Data Initiative – Lightning Talk
CUAHSI HIS Sharing hydrologic data
TIGGE Archives and Access
Web Advisory Group (WAG) Implementation Plan
HYCOM CONSORTIUM Data and Product Servers
IDV Perspective: Climatology of the 2005 Hurricane Season
Prepared by: Jennifer Saleem Arrigo, Program Manager
Data Management Components for a Research Data Archive
Presentation transcript:

Director’s Report Unidata Users Committee Meeting 13 October 2005 Boulder, CO Mohan Ramamurthy Unidata Program Center UCAR Office of Programs Boulder, CO

Unidata System use During Katrina Unidata systems demonstrated their robustness when Hurricane Katrina battered the Gulf Coast, proving invaluable to emergency operations at the Southern Region Climate Center at LSU. The SRCC, an active member of the Unidata community, is a top-tier NOAAport ingest site for the IDD using Unidata's NOAAport satellite ingest software. For analysis and visualization of data, they use GEMPAK a software package upgraded and maintained by the program center. Prior to, during, and following passage of Hurricane Katrina. the SRCC used IDD data to brief emergency managers at the Louisiana Office of Homeland Security and Emergency Preparedness.

Data Distribution About 306 machines in 154 network domains are running LDM-6 and reporting real time statistics in the IDD.   The average hourly volume of data flowing through the IDD has increased to about 3.0 Gigabytes, with a maximum rate of approximately 4.5 Gigabytes. Data Volumes flowing through Internet 2 have experienced a 50% increase over the past year from 14 TB of data to 21 TB now. Recently, RASS data from NOAA Profiler Network has been added to the IDD. UPC working to display the data in viz. tools. RASS: Radio Acoustic Sounding System

Software Development and Usage   LDM released; supports automatic switching between primary and alternate transfer- mode by downstream LDMs – no more need for manual failovers!   GEMPAK: Two new releases since my last report; Latest release 5.8.3a; Downloads of Linux binaries outnumber those for other operating system by a wide margin.   McIDAS Version 2005, released in July, has had downloads from 21 unique sites; in comparison, there have been downloads from 102 unique sites for McIDAS 2004;

Integrated Data Viewer The use of IDV continues to increase. Over the period of July 5 to Sep. 9, 2005, the IDV was used an average of 127 times per [week] day. In comparison, the IDV was used an average of 62 times last year during a similar period. C130 flight track from Airborne Carbon in the Mountains Experiment displayed in the IDV Downloads since March 18: Total unique sites: 617 US: 294 Non-US: 155 Unknown: 168 (IP only) By version: pre1_2: 123 1_2: 210 1_2b2: 451 These numbers are underestimates of the true numbers!

A New Graphing Capability in the IDV The IDV developers have been working closely with NCAR divisions to improve the access to and display of field project data. New graphing capabilities were added for time series, histogram, and scatter plots.

File Formats Local/Remote Services Underlying Interfaces Primary Interfaces THREDDS catalog TDS (THREDDS Data Server Interface) OGC WCS (Web Coverage Server) NetCDF interfaceOPeNDAP OpenDAPADDE GRID Station AREA IOsp GRIB GINI NIDS netCDF via HTTP Jgoffs DMSP NetCDF Common Data Model – TDS

LEAD Update   The LEAD project successfully completed the two-year NSF site review in July. As a result, NSF actually gave additional funding for two new students.   The Unidata LEAD team is designing and building the THREDDS Data Repository, based on a modular framework.   Locally steered WRF running on the Unidata LEAD testbed four times daily;   ADAS (assimilated) hourly fields also made available on the UPC testbed.   For both WRF and ADAS datasets, we are generating THREDDS catalogs and serving the output files via OPeNDAP;   The Unidata LEAD testbed has completed its phase 2 implementation.

40 TB Storage Cluster ~30 GFLOP Processing Cluster Portal Servers for Web, OpenDAP, Grid and LDM Services Unidata LEAD Testbed Beowulf Cluster with Eight Nodes each with Two Athlon CPUs Connected by a Gigabit Fibre Network

LEAD Storage Cluster Nodes Publish Volumes with iSCSI over a Gigabit Network Six Storage Nodes each with Twenty-four 400 GB Hard Drives for Two 4 TB RAID 5 Volumes Gateway Builds and Offers Two 20 TB RAID Volumes using mdadm, LVM, XFS and NFS

Support Load   The total support load at the UPC remains high but steady.   Web-based support system (eSupport) is about two months behind target; Planning to introduce eSupport to the community in a test mode in October.   Support required for data (LDM, IDD) is comparable to that for visualization packages (GEMPAK, IDV, and McIDAS)   The IDV support LOAD continues to grow as more users adopt the package.

2005 Training Workshops  Training Workshops: Annual SoftwareTraining Workshops are offered free of charge once each year.  The workshops are more popular than ever and are over subscribed.  68 participants from 5 countries participated in this year’s workshops on LDM (2 sessions), GEMPAK, McIDAS, and IDV (3 sessions).  Workshop materials are available online and the sessions are webcast live to the community.

SOARS Participation UPC staff continue to be actively involved in SOARS. UPC staff continue to be actively involved in SOARS. Alisha Fernandez-Science Alisha Fernandez-Science Science mentor: Jennifer Oxelson & Lawrence Buja, SCDScience mentor: Jennifer Oxelson & Lawrence Buja, SCD Enhancing access to climate model metadata via a web-accessible database; Shanna-Shaye Forbes Shanna-Shaye Forbes Science mentor: Anne WilsonScience mentor: Anne Wilson Adapting existing decoder software to enable data distribution via InterNet News Three writing/community protégés: Clarence Mann (Writing mentor – Ethan Davis) Nicolle Ngo (Writing mentor – Jo Hansen) Keith Goodman (Community mentor: Jeff Weber)

Unidata’s New Website Plaza Version 2: Launched last week. The second version Unidata’s community web portal features a fresh new look and many new capabilities. A new database schema has been created to address current and future needs. Eventually, e-support will be integrated into the website. The initial response to the new website has been very positive

InstitutionProposal Funded Cost (not including OH) Iowa State University Increasing GIS Web Services Capacity to Serve the Unidata Community $12,000 Louisiana State University IDD Community Infrastructure Enhancement at LSU $15,200 Pennsylvania State University Transition of Unidata IDD Relay Service to a 64-Bit Architecture $7,000 University of Hawaii at Manoa VisionLab 2005 $11,700 University of Missouri-Kansas City Enhancing Meteorological Education and Climate Research, and K-12 Educational Outreach at the University of Missouri-Kansas City $19,000 Purdue University Equipment to Enhance the Capacity to Process and Disseminate Value-added Weather Data at Purdue University $7,000 Western Illinois University New Establishment of Unidata Site for Meteorological Education $4,000 Total$75, Equipment Awards

State of the Program: A Snapshot Community relations: Green Community relations: Green Data flows: Green Data flows: Green Software development: Green Software development: Green Collaborations: Green Collaborations: Green Staffing: Green Staffing: Green Support: Green Support: Green Finances : Yellow (negative cash flow projected for FY06 & FY07) Finances : Yellow (negative cash flow projected for FY06 & FY07)