VLab: Collaborative Grid Services and Portals to Support Computational Material Science Mehmet Nacar, Mehmet Aktas, Marlon Pierce, Zhenyu Lu, Gordon Erlebacher,

Slides:



Advertisements
Similar presentations
LEAD Portal: a TeraGrid Gateway and Application Service Architecture Marcus Christie and Suresh Marru Indiana University LEAD Project (
Advertisements

A Workflow Engine with Multi-Level Parallelism Supports Qifeng Huang and Yan Huang School of Computer Science Cardiff University
ASCR Data Science Centers Infrastructure Demonstration S. Canon, N. Desai, M. Ernst, K. Kleese-Van Dam, G. Shipman, B. Tierney.
This product includes material developed by the Globus Project ( Introduction to Grid Services and GT3.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
1 Introduction to XML. XML eXtensible implies that users define tag content Markup implies it is a coded document Language implies it is a metalanguage.
Office of Science U.S. Department of Energy Grids and Portals at NERSC Presented by Steve Chan.
Brief Overview of Major Enhancements to PAWN. Producer – Archive Workflow Network (PAWN) Distributed and secure ingestion of digital objects into the.
OCT1 Principles From Chapter One of “Distributed Systems Concepts and Design”
Layers & Tiers Umair Javed Lec - 41.
Grids and Grid Technologies for Wide-Area Distributed Computing Mark Baker, Rajkumar Buyya and Domenico Laforenza.
Interpret Application Specifications
Principles for Collaboration Systems Geoffrey Fox Community Grids Laboratory Indiana University Bloomington IN 47404
QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.
Grid Computing, B. Wilkinson, a.1 Grid Portals.
Apache Airavata GSOC Knowledge and Expertise Computational Resources Scientific Instruments Algorithms and Models Archived Data and Metadata Advanced.
 Cloud computing  Workflow  Workflow lifecycle  Workflow design  Workflow tools : xcp, eucalyptus, open nebula.
Application Web Service Toolkit Geoffrey Fox, Marlon Pierce, Ozgur Balsoy Indiana University July
Agent-based Device Management in RFID Middleware Author : Zehao Liu, Fagui Liu, Kai Lin Reporter :郭瓊雯.
Managing Service Metadata as Context The 2005 Istanbul International Computational Science & Engineering Conference (ICCSE2005) Mehmet S. Aktas
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
Chemical Toxicity and Safety Information System Shuanghui Luo Ying Li Jin Xu.
Towards a Javascript CoG Kit Gregor von Laszewski Fugang Wang Marlon Pierce Gerald Guo
DISTRIBUTED COMPUTING
Service Architecture of Grid Faults Diagnosis Expert System Based on Web Service Wang Mingzan, Zhang ziye Northeastern University, Shenyang, China.
Enterprise JavaBeans. What is EJB? l An EJB is a specialized, non-visual JavaBean that runs on a server. l EJB technology supports application development.
第十四章 J2EE 入门 Introduction What is J2EE ?
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
International Telecommunication Union Geneva, 9(pm)-10 February 2009 ITU-T Security Standardization on Mobile Web Services Lee, Jae Seung Special Fellow,
Grids and Portals for VLAB Marlon Pierce Community Grids Lab Indiana University.
Through the development of advanced middleware, Grid computing has evolved to a mature technology in which scientists and researchers can leverage to gain.
RELATIONAL FAULT TOLERANT INTERFACE TO HETEROGENEOUS DISTRIBUTED DATABASES Prof. Osama Abulnaja Afraa Khalifah
The PROGRESS Grid Service Provider Maciej Bogdański Portals & Portlets 2003 Edinburgh, July 14th-17th.
1 Schema Registries Steven Hughes, Lou Reich, Dan Crichton NASA 21 October 2015.
GEM Portal and SERVOGrid for Earthquake Science PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce Computer Science, Informatics, Physics.
Application portlets within the PROGRESS HPC Portal Michał Kosiedowski
Information Builders : SmartMart Seon-Min Rhee Visualization & Simulation Lab Dept. of Computer Science & Engineering Ewha Womans University.
Service - Oriented Middleware for Distributed Data Mining on the Grid ,劉妘鑏 Antonio C., Domenico T., and Paolo T. Journal of Parallel and Distributed.
Middleware for Grid Computing and the relationship to Middleware at large ECE 1770 : Middleware Systems By: Sepehr (Sep) Seyedi Date: Thurs. January 23,
Large Scale Nuclear Physics Calculations in a Workflow Environment and Data Provenance Capturing Fang Liu and Masha Sosonkina Scalable Computing Lab, USDOE.
1 Grid Portal for VN-Grid Cu Nguyen Phuong Ha. 2 Outline Some words about portals in principle Overview of OGCE GridPortlets.
 Apache Airavata Architecture Overview Shameera Rathnayaka Graduate Assistant Science Gateways Group Indiana University 07/27/2015.
Ames Research CenterDivision 1 Information Power Grid (IPG) Overview Anthony Lisotta Computer Sciences Corporation NASA Ames May 2,
Framework for MDO Studies Amitay Isaacs Center for Aerospace System Design and Engineering IIT Bombay.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
6 February 2009 ©2009 Cesare Pautasso | 1 JOpera and XtremWeb-CH in the Virtual EZ-Grid Cesare Pautasso Faculty of Informatics University.
Designing Grid Tag Libraries and Grid Beans Mehmet Nacar, Marlon Pierce, Gordon Erlebacher, Geoffrey Fox GCE 2006 Tampa, FL.
Some comments on Portals and Grid Computing Environments PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce Computer Science, Informatics,
© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/3 JSP Application Models.
Development of e-Science Application Portal on GAP WeiLong Ueng Academia Sinica Grid Computing
In Vivo Imaging Middleware and Applications RSNA 2007 Berkant Barla Cambazoglu The Ohio State University Department of Biomedical Informatics.
AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.
PROGRESS: GEW'2003 Using Resources of Multiple Grids with the Grid Service Provider Michał Kosiedowski.
Rights Management for Shared Collections Storage Resource Broker Reagan W. Moore
Application Web Service Toolkit Allow users to quickly add new applications GGF5 Edinburgh Geoffrey Fox, Marlon Pierce, Ozgur Balsoy Indiana University.
INFSO-RI JRA2 Test Management Tools Eva Takacs (4D SOFT) ETICS 2 Final Review Brussels - 11 May 2010.
Interacting Data Services for Distributed Earthquake Modeling Marlon Pierce, Choonhan Youn, and Geoffrey Fox Community Grids Lab Indiana University.
V7 Foundation Series Vignette Education Services.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
AMSA TO 4 Advanced Technology for Sensor Clouds 09 May 2012 Anabas Inc. Indiana University.
Data Grids, Digital Libraries and Persistent Archives: An Integrated Approach to Publishing, Sharing and Archiving Data. Written By: R. Moore, A. Rajasekar,
Self Healing and Dynamic Construction Framework:
Bringing Grid & Web Services Together
Distribution and components
Metadata-rich Service Oriented Grid Portals
VLab (Virtual Laboratory for Earth and Planetary Materials )
Gordon Erlebacher Florida State University
Presentation transcript:

VLab: Collaborative Grid Services and Portals to Support Computational Material Science Mehmet Nacar, Mehmet Aktas, Marlon Pierce, Zhenyu Lu, Gordon Erlebacher, Dan Kigelman, Evan F. Bollig, Cesar De Silva, Benny Sowell, and David A. Yuen GCE’ November 2005 Seattle, WA

Introduction Grid and Web Service-based system for enabling distributed and collaborative computational chemistry and material science applications for the study of planetary materials. The requirements of The Virtual Laboratory for Earth and Planetary Materials (VLab) include –job preparation and submission, –job monitoring –data storage and analysis –distributed collaboration. These components are divided into client entry (input file creation, visualization of data, task requests) and backend services (storage, analysis, computation). Clients and services communicate through NaradaBrokering We describe two aspects of VLab in this paper: 1.Data entry and submission, 2.Visualization web client/service.

Research Issues VLab presents several interesting problems. Driving Grid research issues include the following: –Persistently managing user inputs as archived, hierarchical project metadata. –Simplifying complicated, multi-staged job submissions and monitoring using Grid portal technology. –Integrating VLab applications with Grid messaging infrastructure to virtualize resource usage, provide fault tolerance, and enable collaboration.

Managing Plane Wave Self Consistent Field (PWSCF) Calculations User session archive management, since VLab submissions may involve dozens or hundreds of simulation runs per user. We also anticipate the addition of more codes from the Quantum Espresso suite and the need to couple these into workflows.

PWSCF Input Forms

JSF Grid Beans Using one bean that is a bean factory for GenericGridTask beans. We create and manage multiple beans for each task. –That is, I submit the job four times in one session. –Similarly, we can create multiple task graph clones. Beans have listeners and maintain state. –Unsubmitted, submitted, active, suspended, resumed are “live” Stored in live repository –Failed, canceled, completed, unknown are “dead” Stored in archive (WS-Context or other) JSF grid beans can be easily serialized with XML. –Castor, XML Beans –Marshal and un-marshal user input for persistent storage in XML storage services OGSA-DAI, GPIR, WS-Context Data model classes handles monitoring of submitted jobs. These classes handled by JSF Data Table element.

Task Management Class Structure

Constructing Task Graphs

Managing Grid Tasks and TaskGraphs Task Manager handles independent user requests, or tasks, from the portlet client in Grid services. –The user request-generating objects are simply Java Bean class instances that wrap common Grid actions (launching remote commands, transferring data, performing remote file operations) using Java COG classes. TaskGraph Manager handles multiple-step task submission –The TaskGraph Manager coordinates user requests with TaskGraph backing beans. –Each TaskGraph bean is itself composed of GenericGridBean implementation instances (for file transfer, job submission, etc.). –Express the dependencies using JSF tag library extensions, so that the JSF application developer can encode the composite task graph workflow out of reusable tags. –The TaskGraph Manager submits and monitors TaskGraphs through an action method when the user launches the job.

TaskGraph Submission Form gdfgfd Fill out the form below (*) fields required Corresponding JSF snippets

Task Monitoring with JSF Data Model Corresponding Java class. public class Job { private String jobId; private String status; private String submitDate; private String finishDate; }

NaradaBrokering Publish/subscribe paradigm, Reliable/robust flexible messaging. Middleware infrastructure is designed around a scalable distributed network of cooperating message routers and processors. NaradaBrokering supports - High-performance Collaborative environments. - Core Web and Grid capabilities Current Pub/Sub Capabilities Multiple transport support Subscription Formats Messaging Related Compliance Current Grid/Web Application Support Reliable delivery Ordered delivery Recovery and Replay Security Message Payload options Grid Application Support Web Services

Collaborative Web Services (1)

Collaborative Web Services (2) A network of three NaradaBrokering brokers is deployed across Indiana University, Florida State University and University of Minnesota. Attached to the network are two wavelet services, with their service adaptors, two schedulers, and several client adaptors. Clients can access the wavelet service via a wavelet applet using standard browsers.

Visualization In our wavelet application, the applet displays the coefficients as spheres centered at the location occupied by the center of the 3D wavelet. High performance graphics are obtained through the use of JOGL, an OpenGL API for Java.

Conclusion and Future Work We have addressed three important components of this framework: data entry, job submission, and backend services. Our work is guided by the principle of ease of use, fault tolerance, collaboration, and persistent records. Our future work will focus on improving the flexibility of collaborative environment Utilizing data flow model to formulate the task execution and providing the directory service to search the available services. The Web Service Resource Framework and particularly the WS- Notification specification family may be used to replace our pre-Web Service topic system. Support for WS-Notification is currently being developed in NaradaBrokering. When this is available we will evaluate its use in our system.

Conclusion Although we include at two entities of each type in our distributed system (2+ schedulers, 2+ wavelet services, etc.), there is as yet no attempt to take network and server load into account when choosing which units will perform the actual work. It is currently a first come first chosen approach. We will evaluate existing work and enhance our schedulers and services to activate themselves based on a more realistic measure of instantaneous or extended load. Collaboration is a natural attribute of our system. Two user tasks that subscribe to identical topics automatically receive the same information. We will investigate approaches to achieve this collaboration both at the visual level (shared user interfaces and displays), with the possibility of multiple users controlling the input. Much work has been done in this area, albeit (to the authors’ knowledge) not within the context of publish/subscribe middleware. Complex workflows are important within VLab. Recent research has shown how to implement strategies for specifying workflows across multiple services. This work will e integrated within our system to properly link input, job submission, analysis, feedback to the user, and finally, automatic (or semi-automatic) decisions regarding the next set of simulations to submit.