IU QuakeSim/E-DECIDER Effort. QuakeSim Accomplishments (1) Deployed, improved many QuakeSim gadgets for standalone integration into QuakeSim.org – Disloc,

Slides:



Advertisements
Similar presentations
Improving Learning Object Description Mechanisms to Support an Integrated Framework for Ubiquitous Learning Scenarios María Felisa Verdejo Carlos Celorrio.
Advertisements

A Workflow Engine with Multi-Level Parallelism Supports Qifeng Huang and Yan Huang School of Computer Science Cardiff University
E-DECIDER: QuakeSim Tools and Products Marlon Pierce, Co-Investigator Margaret Glasscoe, PI
IU QuakeSim Updates Co-PIs: Marlon Pierce, Geoffrey Fox Students and Staff: Xiaoming Gao, Jun Ji, Jun Wang, Marie Ma, Josh Rosen.
QuakeSim Science Gateway: ACES Update Marlon Pierce, Jun Wang, Yu Ma, Jun Ji, Xiaoming Gao, Geoffrey Fox Indiana University.
Progress Report: Hazard Map Generation with OpenSHA E-DECIDER IU Team.
E-DECIDER Workshop: QuakeSim Tools and Products Marlon Pierce Indiana University.
Clouds from FutureGrid’s Perspective April Geoffrey Fox Director, Digital Science Center, Pervasive.
NSDI and Cyberinfrastructure Doug Nebert April 2010.
InSAR Data and GeoServer IU QuakeSim Team October 26, 2011.
Indiana University QuakeSim Activities Marlon Pierce, Geoffrey Fox, Xiaoming Gao, Jun Ji, Chao Sun.
Example Data Products for integration into the Collaborative Common Operating Picture (C-COP): E-DECIDER and GeoGateway National Aeronautics and Space.
DAvinCi: A Cloud Computing Framework for Service Robots
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
INTRODUCTION TO CLOUD COMPUTING Cs 595 Lecture 5 2/11/2015.
CLOUD COMPUTING. A general term for anything that involves delivering hosted services over the Internet. And Cloud is referred to the hardware and software.
A Brief Overview by Aditya Dutt March 18 th ’ Aditya Inc.
Cloud Computing for the Enterprise November 18th, This work is licensed under a Creative Commons.
Discussion and conclusion The OGC SOS describes a global standard for storing and recalling sensor data and the associated metadata. The standard covers.
Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over the Internet. Cloud is the metaphor for.
Connecting OurGrid & GridSAM A Short Overview. Content Goals OurGrid: architecture overview OurGrid: short overview GridSAM: short overview GridSAM: example.
MapReduce April 2012 Extract from various presentations: Sudarshan, Chungnam, Teradata Aster, …
Cloud Computing. What is Cloud Computing? Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable.
Future Grid Future Grid User Portal Marlon Pierce Indiana University.
Distributed Voyager (DVoy) Web Services
An emerging computing paradigm where data and services reside in massively scalable data centers and can be ubiquitously accessed from any connected devices.
CS525: Special Topics in DBs Large-Scale Data Management Hadoop/MapReduce Computing Paradigm Spring 2013 WPI, Mohamed Eltabakh 1.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES High Performance Computing applications in GEON: From Design to Production Dogan Seber.
material assembled from the web pages at
Hadoop/MapReduce Computing Paradigm 1 Shirish Agale.
Introduction to Hadoop and HDFS
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Collaborating with iPlant.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Collaborating with iPlant.
The Limitation of MapReduce: A Probing Case and a Lightweight Solution Zhiqiang Ma Lin Gu Department of Computer Science and Engineering The Hong Kong.
The Future of the iPlant Cyberinfrastructure: Coming Attractions.
(Building the Presentation Layer - KISS). Figuring out what to do! Every was not learnt over night and certainly not without proper help and support.
Workflow Project Status Update Luciano Piccoli - Fermilab, IIT Nov
GEM Portal and SERVOGrid for Earthquake Science PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce Computer Science, Informatics, Physics.
SBIR Final Meeting Collaboration Sensor Grid and Grids of Grids Information Management Anabas July 8, 2008.
 Apache Airavata Architecture Overview Shameera Rathnayaka Graduate Assistant Science Gateways Group Indiana University 07/27/2015.
1 Computing Challenges for the Square Kilometre Array Mathai Joseph & Harrick Vin Tata Research Development & Design Centre Pune, India CHEP Mumbai 16.
9 Systems Analysis and Design in a Changing World, Fourth Edition.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
Web Pages with Features. Features on Web Pages Interactive Pages –Shows current date, get server’s IP, interactive quizzes Processing Forms –Serach a.
Integrating Geographical Information Systems and Grid Applications Marlon Pierce Contributions: Ahmet Sayar,
Exploring ‘Workspaces’ Tom Visser, SARA compute and networking services, Amsterdam Garching Workshop 21 st September 2010.
| nectar.org.au NECTAR TRAINING Module 2 Virtual Laboratories and eResearch Tools.
Fire Emissions Network Sept. 4, 2002 A white paper for the development of a NSF Digital Government Program proposal Stefan Falke Washington University.
Hadoop/MapReduce Computing Paradigm 1 CS525: Special Topics in DBs Large-Scale Data Management Presented By Kelly Technologies
AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.
HPC in the Cloud – Clearing the Mist or Lost in the Fog Panel at SC11 Seattle November Geoffrey Fox
Directions in eScience Interoperability and Science Clouds June Interoperability in Action – Standards Implementation.
Tomlinson’s “System” Design Requirements. 10 Stages Stage 1: Strategic purpose Stage 2: Plan for planning Stage 3: Technology seminar Stage 4: Describe.
Interacting Data Services for Distributed Earthquake Modeling Marlon Pierce, Choonhan Youn, and Geoffrey Fox Community Grids Lab Indiana University.
Breaking the frontiers of the Grid R. Graciani EGI TF 2012.
Esri UC 2014 | Technical Workshop | Address Maps and Apps for State and Local Government Allison Muise Nikki Golding Scott Oppmann.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Our experience with NoSQL and MapReduce technologies Fabio Souto.
Tools and Services Workshop Overview of Atmosphere
The Improvement of PaaS Platform ZENG Shu-Qing, Xu Jie-Bin 2010 First International Conference on Networking and Distributed Computing SQUARE.
Effective way to build test Automation strategy in Agile
University of Technology
Ramesh Baral Team: Marjani Peterson, Andre Guerrero
OGC Testbed 13 Outcomes and Information on the OGC Testbed 14
Lecture 1: Multi-tier Architecture Overview
Cloud Computing: Concepts
Gordon Erlebacher Florida State University
QuakeSim Quarterly Update
Presentation transcript:

IU QuakeSim/E-DECIDER Effort

QuakeSim Accomplishments (1) Deployed, improved many QuakeSim gadgets for standalone integration into QuakeSim.org – Disloc, RDAHMM, Seismicity movie generator (with ANSS data), LOS Tool Deployed GeoServer instance that provides Open Geospatial Consortium programming interfaces to UAVSAR imagery. – Included converting proprietary data formats to GeoTIFF. – Allows us to extract data from imagery through online services, not just make nice pictures. – Provides a solution for some data-centric problems: with subsetting, we don’t need to move large data around. Extended GeoServer with several QuakeSim-specific REST interfaces – Line of Sight services, Simplex input extraction services

QuakeSim Accomplishments (2) Developed a Line of Sight tool for exploring UAVSAR data sets using GeoServer and QuakeTables on the backend. Exploring GeoServer as an Amazon Cloud Service – Need better networking for interactive interfaces – Also integrate with other OGC service instances Several upgrades to the RDAHMM and Simplex to incorporate more data sets. – Access to UNAVCO data sets added to both RDAHMM (time series) and Simplex (velocities) – Complete RDAHMM analysis to the data collected from over 1000 GPS ground stations in Japan during the 2011 Tohoku earthquake with Web front end.

GeoServer and UAVSAR Open Layers Output KML Output Embed into interactive user interfaces: LOS gadget

Fault Refinements with Simplex Simplex uses GPS stations (left) and other surface observations to make refined fault models. This could be used to make better inputs for HAZUS inputs.

RDAHMM Integration with QuakeSim.org

GPS Time Series Plot in Both Absolute Value and Displacements For sustainability, we will convert this Flash interface to DyGraphs (pure JavaScript) before the project completes.

Appendix – web interface for Japan data analysis

Cloud Computing (1) Our initial efforts were to develop a Virtual Block Store system. – Reverse engineer Amazon’s EBS service. – Provides virtual disks, RAID Based on feedback from previous reviews, we focused more effort on practical deliverables. Key problem: data for LOS and Simplex computations is too cumbersome to move around. – We implemented prototypes of these use cases

Cloud Computing (2) Our current GeoServer instance serves as a proof of concept for handling data collections of geophysical imagery data, providing APIs for data access, and demonstrating how to do basic computations near the data. We are now moving GeoServer to Amazon’s cloud – Investigate scalability strategies for cloud-GIS infrastructure on cloud- based data storage. – Build ready-to-use cloud GIS server as a virtual machine instance for rapid distribution of large amount of images and other geo-spatial data – Design and test work flow for geo-spatial data-chaining operations among multiple data distribution agencies. We will evaluate network performance and compare to IU-based services before project completion.

What Applications work in Clouds Pleasingly parallel applications of all sorts analyzing roughly independent data or spawning independent simulations – Long tail of science – Integration of distributed sensors (Internet of Things) Science Gateways and portals Workflow federating clouds and classic HPC Commercial and Science Data analytics that can use MapReduce (some of such apps) or its iterative variants (most analytic apps) 11

Clouds and Grids/HPC Synchronization/communication Performance Grids > Clouds > Classic HPC Systems Clouds appear to execute effectively Grid workloads but are not easily used for closely coupled HPC applications Service Oriented Architectures and workflow appear to work similarly in both grids and clouds Assume for immediate future, science supported by a mixture of – Clouds – see previous application discussion – Grids/High Throughput Systems (moving to clouds as convenient) – Supercomputers (“MPI Engines”) going to exascale

Clouds and QuakeSim Much of QuakeSim is very suitable for high throughput programming paradigm well supported by clouds – e.g. running off many independent image or virtual California runs – Supporting GPS sensors We have researched the unclear data model clouds looking at open source Hbase and HDFS technologies We have also significant work on improving MapReduce which is becoming in some areas dominant data analysis paradigm – Better Scheduling – Iterative Models

Never Hurts to Mention… We (IU) use SourceForge SVN for all code. You can check activity of our code repository either through SourceForge or Ohloh. We switched to JIRA from Bugzilla for task tracking, so you can follow along with what we are doing. – Integrated with SVN: each task corresponds to one or more code check-ins.

E-DECIDER Augmentation Activities

Accomplishments Integrated several QuakeSim tools into E-DECIDER interface. – Simplex, HAZUS Gadget, RSS Disloc Improved reliability, sustainability, and user interface of RSSDisloc Converted desktop OpenSHA tools to online REST services – This was a relatively time-consuming effort – Need however better connections with OpenSHA group. – Recommend Apache-style open processes

RSS Disloc Calculates surface displacements, synthetic InSAR from the USGS M>5 Earthquake RSS feed. Can by integrated with OpenSHA HAZUS input generation tools.

HAZUS Gadgets Download results for now but could wrap in KML Can generate HAZUS input files interactively.

Synergistic Work From the E-DECIDER workshop (October 2011), it was clear that GIS tools to make imagery data available – GeoServer + UAVSAR GeoTIFF data collections are a potential resource for the community. – We have shown in QuakeSim how to extend with REST interfaces to make mash-ups. Need cultural changes for better collaborations with other “open source” projects like OpenSHA. – Open Community, not open source