OGSA-DAI User Requirements and Scenarios

Slides:



Advertisements
Similar presentations
1 OGSA-DAI Platform Dependencies Malcolm Atkinson for OMII SC 18 th January 2005.
Advertisements

Eldas 1.0 Enterprise Level Data Access Services Design Issues, Implementation and Future Development Davy Virdee.
Welcome to Middleware Joseph Amrithraj
An Overview of OGSA-DAI Kostas Tourlas
By Adam Balla & Wachiu Siu
Amy Krause Applications Consultant, EPCC Tom Sugden Applications Consultant, EPCC OGSA-DAI Usage Scenarios Behind.
17 July 2006ISSGC06, Ischia, Italy1 Agenda Session 26 – 14:30-16:00 An Overview of OGSA-DAI OGSA-DAI today – and future features How to extend OGSA-DAI.
SaaS, PaaS & TaaS By: Raza Usmani
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
Microsoft ® Application Virtualization 4.6 Infrastructure Planning and Design Published: September 2008 Updated: February 2010.
VAP What is a Virtual Application ? A virtual application is an application that has been optimized to run on virtual infrastructure. The application software.
1 Dr. Markus Hillenbrand, ICSY Lab, University of Kaiserslautern, Germany A Generic Database Web Service for the Venice Service Grid Michael Koch, Markus.
Unit – I CLIENT / SERVER ARCHITECTURE. Unit Structure  Evolution of Client/Server Architecture  Client/Server Model  Characteristics of Client/Server.
Computer Emergency Notification System (CENS)
OGSA-DAI Scenarios and Requirements OGSA-DAI for Developers GridWorld 2006, Washington DC 11 September 2006.
OGSA-DAI in OMII-Europe Neil Chue Hong EPCC, University of Edinburgh.
WebServices, GridServices and Firewalls Matthew J. Dovey Technical Manager Oxford e-Science Centre
1 1 EPCC 2 Curtin Business School & Edinburgh University Management School Michael J. Jackson 1 Ashley D. Lloyd 2 Terence M. Sloan 1 Enabling Access to.
1 MSCS 237 Overview of web technologies (A specific type of distributed systems)
® IBM Software Group © 2007 IBM Corporation Best Practices for Session Management
OGSA-DAI.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
Amy Krause EPCC OGSA-DAI An Overview OGSA-DAI Technology Update GGF17, Tokyo (Japan)
CEOS Working Group on Information Systems and Services - 1 Data Services Task Team Discussions on GRID and GRIDftp Stuart Doescher, USGS WGISS-15 May 2003.
Neil Chue Hong Project Manager, EPCC OGSA-DAI Requirements Gathering Exercise 2 nd DIALOGUE workshop eSI, 9-10.
Dispatching Java agents to user for data extraction from third party web sites Alex Roque F.I.U. HPDRC.
E-commerce Architecture Ayşe Başar Bener. Client Server Architecture E-commerce is based on client/ server architecture –Client processes requesting service.
OGSA-DAI.
Amy Krause EPCC OGSA-DAI An Overview OGSA-DAI on OMII 2.0 OMII The Open Middleware Infrastructure Institute NeSC,
System Software Laboratory Databases and the Grid by Paul Watson University of Newcastle Grid Computing: Making the Global Infrastructure a Reality June.
Distributed Systems Architectures. Topics covered l Client-server architectures l Distributed object architectures l Inter-organisational computing.
Progress Apama Fundamentals
Happy Endings: Reengineering Wesleyan’s Software Deployment to Labs and Classrooms Kyle Tousignant 03/22/2016.
Data Virtualization Tutorial… SSL with CIS Web Data Sources
Chapter 8 Environments, Alternatives, and Decisions.
By: Raza Usmani SaaS, PaaS & TaaS By: Raza Usmani
Apache web server Quick overview.
The Client-Server Model
Netscape Application Server
Node.js Express Web Applications
Section 13 - Integrating with Third Party Tools
Overview – SOE PatchTT November 2015.
Outline SOAP and Web Services in relation to Distributed Objects
Overview – SOE PatchTT December 2013.
Logo here Module 3 Microsoft Azure Web App. Logo here Module Overview Introduction to App Service Overview of Web Apps Hosting Web Applications in Azure.
Security Engineering.
Chapter 18 MobileApp Design
The Improvement of PaaS Platform ZENG Shu-Qing, Xu Jie-Bin 2010 First International Conference on Networking and Distributed Computing SQUARE.
Outline SOAP and Web Services in relation to Distributed Objects
Introduction to Networks
Introduction to J2EE Architecture
RIM Blackberry at SAP Key Benefits / Survey Results April 2004 SAP IT Enterprise Telecommunication Services.
PHP / MySQL Introduction
#01 Client/Server Computing
File Transfer Olivia Irving and Cameron Foss
Systems Analysis and Design 5th Edition Chapter 8. Architecture Design
Database Management System (DBMS)
Dev Test on Windows Azure Solution in a Box
POOL/RLS Experience Current CMS Data Challenges shows clear problems wrt to the use of RLS Partially due to the normal “learning curve” on all sides in.
Relate to Clients on a business level
Lecture 1: Multi-tier Architecture Overview
Service Oriented Architecture (SOA)
Unit# 5: Internet and Worldwide Web
Introduction of Week 11 Return assignment 9-1 Collect assignment 10-1
Middleware, Services, etc.
Grid Systems: What do we need from web service standards?
PerformanceBridge Application Suite and Practice 2.0 IT Specifications
Introduction of Week 5 Assignment Discussion
#01 Client/Server Computing
L. Glimcher, R. Jin, G. Agrawal Presented by: Leo Glimcher
Presentation transcript:

OGSA-DAI User Requirements and Scenarios OGSA-DAI Tutorial GGF17, Tokyo

Outline Let’s talk to the users How to use OGSA-DAI productively Who wants to use OGSA-DAI? What do they want to use it for? Why aren’t they using it right now? Who is using OGSA-DAI? What are they using it for? How could they use it more effectively? Who was using OGSA-DAI? Why aren’t they using it now? How to use OGSA-DAI productively 11 May 2006 http://www.ogsadai.org.uk/

Requirements – why and what Learn more about the data access and integration challenges that other projects face Use this information to inform the future development of OGSA-DAI Associate requirements with projects and aid work prioritisation Do what we think most users want VS doing what specific users want What? Data Structure, quantity and types of data resource Queries Types of queries that are performed against this data, query languages, typical size of result sets Problems Data access and integration problems faced What can or could OGSA-DAI provide? Timescale – Nov 2005 to Jan 2006 Rather than guessing what projects want – go out ask and engage with them Provide focus for the team – specific “customers” See if there are common requirements spanning a number of projects that we could address 11 May 2006 http://www.ogsadai.org.uk/

Requirements – who AstroGrid Automed and ISpider CancerGrid ESSC Gold (www.astrogrid.org) – distributed queries over large astronomy databases Automed and ISpider (www.doc.ic.ac.uk/automed) and (www.ispider.man.ac.uk) – model-based data integration and Grid-based informatics platform for proteomics CancerGrid (www.cancergrid.org) – storage and analysis of distributed data containing clinical trial and lab data ESSC (www.nerc-essc.ac.uk) – environmental and atmospheric simulations Gold (www.goldproject.ac.uk) – provides infrastructure for virtual organisations NTRAC (www.ntrac.org.uk) – similar to CancerGrid 11 May 2006 http://www.ogsadai.org.uk/

Users want… Efficient bulk data transport Between heterogeneous data resources Required by application-level projects Benefits higher-level middleware (DQP, data federation, etc.) Data federation and distributed query processing across heterogeneous data resources Asynchronous query model Process large, long-running queries Client can poll or be notified of the query status Terminate queries at an intermediate stage Currently focus of performance work and activity framework redesign Currently working on swissSQL activity 11 May 2006 http://www.ogsadai.org.uk/

…and also… Data resource view creation and management Provide different views of data resources to different users in a secure, DBMS-independent manner Manage these views dynamically Security / certificate delegation Access data from other networks with role-based access rules Usability Quick and easy installation, configuration and maintenance Support deployment as a WAR Reduce third-party dependencies or prerequisites Work on dynamic resource creation assists here a view can be viewed as a dynamic resource. Now bundle many more third-party JARs to reduce download overheads 11 May 2006 http://www.ogsadai.org.uk/

Now what… Focus on high-priority requirements raised by projects Continued scenario-driven development: Project has a specific well-defined data access or integration scenario Can OGSA-DAI support that scenario? Yes? Almost? What are OGSA-DAI’s limitations and how can these be addressed? No? What functionality is needed within OGSA-DAI? Can we spare a developer to work with this project? 11 May 2006 http://www.ogsadai.org.uk/

Usage scenarios “I have a data-related problem and OGSA-DAI made things worse” OGSA-DAI is not a solution to every data access and integration problem in existence “OGSA-DAI is not as fast as JDBC” A 747 is not as fast as an F15 Different products for different requirements OGSA-DAI is like any tool It has strengths and weaknesses There are scenarios where it will be helpful and where it will not Publish usage scenarios Best practice on how to use OGSA-DAI effectively From the experiences of both users and ourselves Current scenarios are on the WWW site An occasional complaint But can you carry 100s of people in a F15 and where is the catering, TV, restroom, luggage space? Scenarios from users provides a way of eliciting the effective, and less effective OGSA-DAI deployments Current scenarios are on the WWW site – I’ll present general ones. More specific ones e.g. for handling BLOBs are on the site 11 May 2006 http://www.ogsadai.org.uk/

A naïve usage 11 May 2006 http://www.ogsadai.org.uk/ OGSA-DAI introduces an additional level of indirection to the data This is good – location transparency and limited database transparency BUT if these are not as important as performance then you get performance degradation  Additional transfer of data via SOAP over HTTP 11 May 2006 http://www.ogsadai.org.uk/

A more effective usage 11 May 2006 http://www.ogsadai.org.uk/ In the upper diagram OGSA-DAI's deliverToURL activity is being used to deliver the data a client's FTP server. The data will therefore be written to a file in the client's own file space (push) In the lower diagram OGSA-DAI deliverToStream activity delivers the data to a servlet from which the client can directly pull the data using HTTP (pull) Two data-loaded SOAP trips are avoided and data no longer needs to pass through the application specific service The client is unaware of OGSA-DAI – it just receives data at its FTP server or accesses a HTTP connection 11 May 2006 http://www.ogsadai.org.uk/

A more effective usage As the data no longer flows back through the application-specific service Provide additional OGSA-DAI activities to do application-specific data processing Configure the OGSA-DAI service to support these activities OGSA-DAI provides the delivery activities out-of-the-box Overhead of developing application-specific data processing is reduced Especially if you wish to experiment Different delivery options Allowing clients to select the desired delivery option 11 May 2006 http://www.ogsadai.org.uk/

Multiple distributed resources Hides DBMS-specific details from the application-specific service Location and (limited) DBMS transparency – client and application-specific services only needs to know resource IDs Not connection URLs or passwords Again data here goes from OGSA-DAI directly to the client removing an additional trip If the OGSA-DAI service is providing more functionality than simple access to a single data resource then its role is crucial 11 May 2006 http://www.ogsadai.org.uk/

Data federation 11 May 2006 http://www.ogsadai.org.uk/ OGSA-DAI can also manage federations of data resources – removes the burden from an application-specific service Relational multi resources, provided in release 2.2, provide support for a simple form of this style of deployment (for multiple relational resources) which, as application developers, you could customise A single resource performs the federation – the client and application-specific service is unaware of the individual federated resources Also possible, in future, to make the application-specific service explicitly aware of the resources – they manage the federation explicitly if additional control is needed 11 May 2006 http://www.ogsadai.org.uk/

Exploiting OGSA-DAI activities Preceding scenarios delegate much application-level functionality to OGSA-DAI so… …why not implement all application-specific functionality as OGSA-DAI activities? Potentially moves computation closer to data Eliminates expensive data movement Improved range of delivery methods A customised OGSA-DAI service can expose only application-specific activities Relocate application-specific functionality from the application-specific service to OGSA-DAI Remove a server-side layer again Use OGSA-DAI client toolkit to construct our Perform document Raw data accessing activities such as as sqlQueryStatement do not have to be exposed to clients 11 May 2006 http://www.ogsadai.org.uk/

What are your requirements? Please do get in touch with the OGSA-DAI team Discuss OGSA-DAI matters Discuss requirements of a specific project Arrange visits and collaborations Contribute your own extensions Feedback and comments are always welcome! Engage in discussions on the OGSA-DAI user list http://www.ogsadai.org.uk 11 May 2006 http://www.ogsadai.org.uk/