A PPARC funded project AstroGrid Architecture Consortium Meeting, Leicester, 3 rd Nov 2003.

Slides:

Advertisements

Similar presentations

Registry Interop Summary IVOA Interoperability meeting Cambridge, Boston, MA May 2004.

Advertisements

May IVOA Interop Meeting1 STScI/JHU Registry Status Gretchen Greene Wil OMullane T HE US N ATIONAL V IRTUAL O BSERVATORY.

September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.

September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.

8 September 2008NVO Summer School 2008 – Santa Fe1 Publishing Data and Services to the VO Ray Plante Gretchen Greene T HE US N ATIONAL V IRTUAL O BSERVATORY.

Abstraction Layers Why do we need them? –Protection against change Where in the hourglass do we put them? –Computer Scientist perspective Expose low-level.

A PPARC funded project AstroGrid Framework Consortium meeting, Dec 14-15, 2004 Edinburgh Tony Linde Programme Manager.

A PPARC funded project Common Execution Architecture Paul Harrison AstroGrid consortium meeting Edinburgh UK Dec 2004.

Building FHIR Servers on Existing Applications

Andrew McNab - EDG Access Control - 14 Jan 2003 EU DataGrid security with GSI and Globus Andrew McNab University of Manchester

14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn.

A PPARC funded project Tony Linde Programme Manager eScience meets eFrameworks 28 th April 2006 NeSC, Edinburgh.

MySpace: Personalized Work Space in AstroGrid Catherine Ling Qin 26 th Nov 2003.

Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.

1 The IIPC Web Curator Tool: Steve Knight The National Library of New Zealand Philip Beresford and Arun Persad The British Library An Open Source Solution.

PAWN: A Novel Ingestion Workflow Technology for Digital Preservation

PAWN: A Novel Ingestion Workflow Technology for Digital Preservation Mike Smorul, Joseph JaJa, Yang Wang, and Fritz McCall.

7.1 © 2004 Pearson Education, Inc. Exam Managing and Maintaining a Microsoft® Windows® Server 2003 Environment Lesson 7: Introducing Group Accounts.

AstroGrid Group 7: Teemu Toivola Tero Viitala. Problem several separate databases no common interface between databases difficulties of joining related.

Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.

Data Grid Web Services Chip Watson Jie Chen, Ying Chen, Bryan Hess, Walt Akers.

FHIRFarm – How to build a FHIR Server Farm (quickly)

System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.

VxOware Progress Report August How to create a new section? Configure section –Create metadata structure (template) –Create elements map for web.

MEDIN Data Guidelines. Data Guidelines Documents with tables and Excel versions of tables which are organised on a thematic basis which consider the actual.

Astrogrid Resource Registry Querying the Registry 1.Mullard Space Science Laboratory, University College London, Holmbury St. Mary, Dorking, Surrey RH5.

5.1 © 2004 Pearson Education, Inc. Lesson 5: Administering User Accounts Exam Microsoft® Windows® 2000 Directory Services Infrastructure Goals 

A PPARC funded project AstroGrid: new technology for the virtual observatory SC2004 Pittsburgh, PA November 2004 Guy Rixon AstroGrid Technical Architect.

Robert Fourer, Jun Ma, Kipp Martin Copyright 2006 An Enterprise Computational System Built on the Optimization Services (OS) Framework and Standards Jun.

AstroGrid Datacenters ESO M Hill (ROE), Aug 2004.

Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.

AstroGrid AstroGrid increases scientific research possibilities by : enabling access to distributed astronomical data and information resources. providing.

Last News of and

A PPARC funded project AstroGrid approach to the Virtual Observatory Architecture ADASS XIV Pasadena Oct-2004 Tony Linde Andrew Lawrence Keith Noddle.

Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.

INFSO-RI Enabling Grids for E-sciencE Logging and Bookkeeping and Job Provenance Services Ludek Matyska (CESNET) on behalf of the.

AstroGrid Overview AG-SAG Cambridge IoA 19 th June 2003 Tony Linde AstroGrid Project Manager University of Leicester, Dept. Physics & Astronomy.

Summary of distributed tools of potential use for JRA3 Dugan Witherick HPC Programmer for the Miracle Consortium University College.

1 All-Hands Meeting 2-4 th Sept 2003 e-Science Centre The Data Portal Glen Drinkwater.

© Wiley Inc All Rights Reserved. MCSE: Windows Server 2003 Active Directory Planning, Implementation, and Maintenance Study Guide, Second Edition.

1 Schema Registries Steven Hughes, Lou Reich, Dan Crichton NASA 21 October 2015.

1 27-Apr-2004AstroGrid Consortium Meeting - PL talk Agenda Tuesday status review / goals / science Lunch + AGLI Itn-05 review Dinner at Coton (share.

Database Design and Management CPTG /23/2015Chapter 12 of 38 Functions of a Database Store data Store data School: student records, class schedules,

Web: Minimal Metadata for Data Services Through DIALOGUE Neil Chue Hong AHM2007.

The Project The Virtual Observatory Technical Progress Andy Lawrence Nottingham All-Hands meeting Sept 2003 AstroGrid

A PPARC funded project Workflow and Job Control in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester.

Policy Based Data Management Data-Intensive Computing Distributed Collections Grid-Enabled Storage iRODS Reagan W. Moore 1.

NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.

Solar and space physics datasets within a Virtual Observatory: the AstroGrid experience Silvia Dalla * and Nicholas A Walton  * School of Physics & Astronomy,

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Aurélien Stébé Registry and Curation, Oct 2005, ESAC, Spain ESAVO Registry.

16 October 2003Registry Interface CallsIVOA Interoperability, Strasbourg IVOA Interoperability Elizabeth Auden & Registry Workgroup 16 – 17 October 2003.

1 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen The Virtual Observatory The VObs concept The structure of the VObs Standards, Standards, Standards.

A PPARC funded project Common Execution Architecture Paul Harrison IVOA Interoperability Meeting Cambridge MA May 2004.

IVOA RM, VOResources, Identifiers, Interfaces Chenzhou CUI.

A PPARC funded project Astro-Wise meeting April 2004 OmegaCEN, Kapteyn Institute, Groningen Tony Linde AstroGrid Project Manager University of Leicester.

System/SDWG Update Management Council Face-to-Face Flagstaff, AZ August 22-23, 2011 Sean Hardman.

DSpace System Architecture 11 July 2002 DSpace System Architecture.

AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.

VO Data Access Layer IVOA Cambridge, UK 12 May 2003 Doug Tody, NRAO.

A PPARC funded project PM Report Grid Steering Committee MRC, London 18-Feb-2004.

1 IVOA Registry Interoperability Meeting Strasbourg 16-Oct-2003.

E-commerce Architecture Ayşe Başar Bener. Client Server Architecture E-commerce is based on client/ server architecture –Client processes requesting service.

IPDA Registry Definitions Project Dan Crichton Pedro Osuna Alain Sarkissian.

Introduction: AstroGrid increases scientific research possibilities by enabling access to distributed astronomical data and information resources. AstroGrid.

Developing our Metadata: Technical Considerations & Approach Ray Plante NIST 4/14/16 NMI Registry Workshop BIPM, Paris 1 …don’t worry ;-) or How we concentrate.

Using iRODS with the EnginFrame Grid Portal into the GRIDA3 project Francesco Locunto Marco Piras Matteo Vocale.

MSSL Astrogrid Workshop

An Overview of Data-PASS Shared Catalog

SDMX IT Tools SDMX Registry

Presentation transcript:

A PPARC funded project AstroGrid Architecture Consortium Meeting, Leicester, 3 rd Nov 2003

2 03-Nov-2003AstroGrid Architecture Agenda Goals reminder General points Approach Registries & Identifiers Components Next? Note: presentation template

3 03-Nov-2003AstroGrid Architecture …

4 03-Nov-2003AstroGrid Architecture Goals A working datagrid for key UK databasesOK: data centres to install AstroGrid High throughput datamining facilities for interrogating those databases ??: GDW A uniform archive query and data-mining software interfaceOK: DataAccess, GDW The ability to browse simultaneously multiple datasetsOK A set of tools for integrated on-line analysis of extracted dataOK: more in AG2 A set of tools for on-line database analysis and explorationOK: more in AG2 A facility for users to upload code to run their own algorithms on the datamining machines ??: AG2 / GRID An exploration of techniques for open-ended resource discoveryOK: Registry

5 03-Nov-2003AstroGrid Architecture Goals: additional Workflow construction Distributed virtual storage space Data access rights User identification, profiling, … Group permissioning Effective usable GUI Complete VO infrastructure

6 03-Nov-2003AstroGrid Architecture …

7 03-Nov-2003AstroGrid Architecture General: Approach Infrastructure Open Interoperability Component-based Deployment Leading-edge Incremental development

8 03-Nov-2003AstroGrid Architecture General: Approach: Infrastructure Other VO projects: Tools: SkyNode, Cone Search, … Registry AstroGrid MySpace Community Workflow Portal Access Security

9 03-Nov-2003AstroGrid Architecture General: Approach: Open Open Source Implement IPL/OPL & Copyright notices Upload releases to nescforge Deployment choices Choice of component installation Open interfaces Server / Browser access Run from anywhere Open Science Free access to all documents Open engagement

10 03-Nov-2003AstroGrid Architecture General: Approach: Interoperability IVOA engagement Resource Registry  Content  Identifiers  Schema ADQL Deployment Registry content SkyNode & ADQL

11 03-Nov-2003AstroGrid Architecture General: Approach: Interoperability Lead IVOA Data access Access policy Authentication & Authorisation Workflow Component interoperability Community MySpace (VOSpace)VOSpace Grid-based components

12 03-Nov-2003AstroGrid Architecture General: Approach: Component-based Plug-n-Play 1. Install 2. Configure 3. Operate Pick-n-Mix Choose components for environment Mix-n-Match Use components from other VO projects Goal: Minimum component interaction Asynchronous calls Call once only Pass maximum info

13 03-Nov-2003AstroGrid Architecture General: Approach: Deployment

14 03-Nov-2003AstroGrid Architecture General: Approach: Leading-edge Not bleeding-edge Latest (mostly proven) technologies Cocoon XML messaging XQuery SOAPy Beans (?) Eclipse IDE even if only by us

15 03-Nov-2003AstroGrid Architecture General: Approach: Incremental development Quarterly releases Weekly integration  Daily Write Test Commit Build Integrate

16 03-Nov-2003AstroGrid Architecture …

17 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers Registries Resource Community MySpace Identifiers AuthorityID ResourceKey Others: datasets, files, tables, …

18 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers: Registries Resource Data: archive, catalog, spectra, … Service: ace, topcat, vot-merge, … Others: registry, authority, … Community Person Group MySpace Items: files, tables, …

19 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers: Identifiers XML structure: AuthorityID ResourceKey IVOA standard String equivalent: ivo://AuthorityID/ResourceKey

20 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers: Identifiers Sub-identifier Extend XML structure Add to string after:  #  ? ivo://AuthorityID/ResourceKey#... ivo://AuthorityID/ResourceKey?... Ignored by resource registry!!

21 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers: AuthorityID Like domain name: astrogrid.org community.astrogrid.org sdss.jhu vizier.u-strasbg.fr Owned by: Organisation Person Registry Allocated to resource by one registry

22 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers: ResourceKey Like path name: J/A+A/385/87 services/SIAP 1XMM Chosen by user Unique across VO Permanent Obsolete names flagged

23 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers: Community – Person & Group Identity same as resource: community.star.le.ac.uk Tony.Linde community.astrogrid.org groups/AGLI u-strasbg.fr Genova Replication? User can choose details to be replicated Default?  Identity, name, affiliation,

24 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers: MySpace Extend resource identifier As string: vosp://community.star.le.ac.uk/Tony.Linde# pub/documents/articles/Strategies for AstroGrid community.star.le.ac.uk Tony.Linde pub/documents/articles Strategies for AstroGrid

25 03-Nov-2003AstroGrid Architecture General: Registries & Identifiers: MySpace AuthorityID  Community registry ResourceKey  Person Person metadata: msManager resource msManager  msRegistry Metadata for all items:  Physical location of msServer msServer Maps name to internal representation Mediates access to item Allocates & initiates transfer, add, delete… community.star.le.ac.uk Tony.Linde pub/documents/articles Strategies for AstroGrid

26 03-Nov-2003AstroGrid Architecture …

27 03-Nov-2003AstroGrid Architecture Component Architecture Registry Security Community Portal Workflow MySpace DataAccess Applications Messaging

28 03-Nov-2003AstroGrid Architecture Components: Registry Schema IVOA Working Draft: RegWp03RegWp03 V0.9: still waiting Extensions?  SkyNode  DataAccess components  Services: TopCat, ACE, … Harvesting: OAI http GET

29 03-Nov-2003AstroGrid Architecture Components: Registry IVOA commitment Use schema Replicate with other registries  VizieR  NVO  others Fully working by 01-Jan-2004 NVO demos at AAS, Atlanta, 4-8 th Jan Remote access via browser Add/Update/Delete resources via GUI

30 03-Nov-2003AstroGrid Architecture Components: Registry: Issues Storage XML native XML via RDBMS RDBMS Query XQuery  Real XQuery (FLWOR) rather than XPath SQL

31 03-Nov-2003AstroGrid Architecture Components: Security Resource has access policy attached Part of metadata (schema extension) Written using ODRL?ODRL Policy identifies Access rights (RWED?) Default: RE Per Group Members inherit Group permissions Group may restrict members’ permissions

32 03-Nov-2003AstroGrid Architecture Components: Security Service call / Data query: Includes user identity & permissions Certification / Encryption Look at GSI User may upload certificate Components have own certificates Component exchange based on:  Component certificate  User certificate

33 03-Nov-2003AstroGrid Architecture Components: Security: Issues Certification implementation Component ‘trust’ Fallback Unidentified user Non-trusted component

34 03-Nov-2003AstroGrid Architecture Components: Community Community registry Relationship to MySpace Groups Contain people (PersonID) Creator has full admin rights over group  Can delegate rights to others Including people from another community ‘Community’ is meaningless Only people & groups have meaning, IDs

35 03-Nov-2003AstroGrid Architecture Components: Community: Issues Registry schema extensions Registry implementation From resource registry? Replication of user metadata If? How?

36 03-Nov-2003AstroGrid Architecture Components: Portal: Issues Adding application links Integrating applets Describing application GUIs XForms Service metadata: pointer to xforms structure Scripting application GUIs XForms?

37 03-Nov-2003AstroGrid Architecture Components: Workflow Add component from tasklist to workflow Appropriate GUI presented  Set parameters  Design query … Save & Resubmit Templates Task groups  Add to workflow Change parameters  Query  Number of records BPEL description

38 03-Nov-2003AstroGrid Architecture Components: Workflow Add generic resource JES selects optimum resources at:  Submission  Run time Task I/O links Identify mismatch Insert converter Progress tracking User notification Use profile in community registry

39 03-Nov-2003AstroGrid Architecture Components: Workflow: Issues Effective GUI (non drag-n-drop) Adding services to ‘toolbar’ Discover via registry Standard + user-selectable?

40 03-Nov-2003AstroGrid Architecture Components: MySpace Identifier as per IVOA standard, extended msManager  msRegistry msServer manages items Mediates access Provides access method

41 03-Nov-2003AstroGrid Architecture Components: MySpace: Issues Access via msServer Access rights Identifier valid for database tables?

42 03-Nov-2003AstroGrid Architecture Components: DataAccess Common interface Plugins for databases

43 03-Nov-2003AstroGrid Architecture Components: DataAccess: Issues FITS file access Wizard to set up service

44 03-Nov-2003AstroGrid Architecture Components: Applications Web service interface/wrapper Should not wrap: Existing web services http GET services Any other callable service

45 03-Nov-2003AstroGrid Architecture Components: Applications: Issues Describing GUI for apps Adding GUI-based services to Portal

46 03-Nov-2003AstroGrid Architecture Components: Messaging Message Queue Message Log Issues?

47 03-Nov-2003AstroGrid Architecture …

48 03-Nov-2003AstroGrid Architecture Next? Complete architecture docs Revise after today Update with interfaces Add use cases & sequence diags Get feedback & complete