Presentation is loading. Please wait.

Presentation is loading. Please wait.

A PPARC funded project AstroGrid and OGSA-DAI: Early experiences Kona Andrews Guy Rixon Institute of Astronomy University of Cambridge.

Published byAllison McCormick Modified over 10 years ago

Similar presentations

Presentation on theme: "A PPARC funded project AstroGrid and OGSA-DAI: Early experiences Kona Andrews Guy Rixon Institute of Astronomy University of Cambridge."— Presentation transcript:

1 A PPARC funded project AstroGrid and OGSA-DAI: Early experiences Kona Andrews Guy Rixon Institute of Astronomy University of Cambridge

2 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Data Set Access: The DataCenter DataCenter component Data Access Plug-in Data Web Services Tomcat / Axis

3 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences DataCenter JDBC plug-in DataCenter component JDBC DBMS Web Services Tomcat / Axis

4 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences DataCenter custom plug-in DataCenter component File querier FITS files Web Services Tomcat / Axis

5 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences DataCenter proxy plug-in DataCenter component Service proxy Vizier Web Services Tomcat / Axis

6 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences What we imagined OGSA- DAI Web Services DataCenter Component OGSA-DAI client plug-in Tomcat / Axis Grid Service Ports Tomcat / OGSA

7 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Problem OGSA- DAI Web Services DataCenter Component OGSA-DAI client plug-in Tomcat / Axis Grid Service Ports Tomcat / OGSA

8 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Problem OGSA- DAI Web Services DataCenter Component OGSA-DAI client plug-in Tomcat / Axis Grid Service Ports Tomcat / OGSA

9 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Nasty solution OGSA- DAI Web Services DataCenter Component OGSA-DAI client plug-in Tomcat / Axis Grid Ports Tomcat / OGSA OGSA-DAI client Separate JVM

10 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Current use of OGSA-DAI activities Data Center OGSA-DAI Postgres DBMS ADQL

11 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Current use of OGSA-DAI activities Data Center OGSA-DAI Postgres DBMS ADQL to SQL ADQL SQL

12 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Current use of OGSA-DAI activities Data Center OGSA-DAI Postgres DBMS ADQL to SQL sqlQueryStatement ADQL SQL

13 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Current use of OGSA-DAI activities Data Center OGSA-DAI Postgres DBMS ADQL to SQL sqlQueryStatement ADQL SQL Result rows SQL (to XML RowSet)

14 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Current use of OGSA-DAI activities Data Center OGSA-DAI Postgres DBMS ADQL to SQL sqlQueryStatement deliverFromURL (to XML RowSet) ADQL SQL Result rows SQL

15 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Current use of OGSA-DAI activities Data Center OGSA-DAI Postgres DBMS ADQL to SQL sqlQueryStatement xslTransform deliverFromURL (to XML RowSet) ADQL SQL Result rows SQL VOTable

16 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Current use of OGSA-DAI activities Data Center OGSA-DAI Postgres DBMS ADQL to SQL sqlQueryStatement xslTransform deliverToFile / deliverToGFTP deliverFromURL (to XML RowSet) ADQL SQL VOTable Result rows SQL

17 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Current use of OGSA-DAI activities Data Center OGSA-DAI Postgres DBMS ADQL to SQL sqlQueryStatement xslTransform deliverToFile / deliverToGFTP deliverFromURL (to XML RowSet) ADQL SQL VOTable Result rows SQL Deliver results

18 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Comments Astronomers often need LARGE result sets Thousands to millions of rows Memory usage is a big issue OGSA-DAI 3.1 much better than 3.0 xslTransform activity hogging memory

19 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences OGSA-DAI speed penalty Rough figures only Measured without xslTransform activity Will likely vary with available memory

20 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Planned extension: User tables OGSA- DAI DBMS Data Center R/O Datasets QueryQuery results

21 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Planned extension: User tables OGSA- DAI DBMS Data Center R/O Datasets User tables QueryQuery resultsData flow GDS to GDS transfer Bulk load OGSA- DAI

22 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences DBMS Planned extension: User tables OGSA- DAI DBMS Data Center Data Center OGSA- DAI Table Manager R/O Datasets User tables QueryQuery resultsData flow

23 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences DBMS Planned extension: User tables OGSA- DAI DBMS Data Center Data Center OGSA- DAI Table Manager Client User data R/O Datasets User tables QueryQuery resultsData flow

24 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Comments Will be relying on sqlBulkLoadRowSet (!) Table management facilities currently thin Not using X.509 auth at user granularity Still require W/S-based access to G/S

25 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences Our needs Ability to handle very large result sets - Dont rely on loading whole result set into memory! Suite of DB/table management activities - List tables in DB - Describe table column types / space usage - Last access/modification time - Etc etc etc Ability to create and delete indices on tables Ultra-simple installation procedures

26 07-04-2004 OGSA-DAI UG Meeting AstroGrid and OGSA-DAI: Early experiences The End www.astrogrid.org

Download ppt "A PPARC funded project AstroGrid and OGSA-DAI: Early experiences Kona Andrews Guy Rixon Institute of Astronomy University of Cambridge."

Similar presentations

Instant JChem - current status and what's coming soon. Tim Dudgeon Solutions for Cheminformatics.

Instant JChem - current status and what's coming soon. Tim Dudgeon Solutions for Cheminformatics.
IVOA, Pune India September 27-29 20041 Data Access Layer Working Group Pune Workshop Summary Doug Tody National Radio Astronomy Observatory International.

IVOA, Pune India September Data Access Layer Working Group Pune Workshop Summary Doug Tody National Radio Astronomy Observatory International.
2003 May 24Clive Page Simple ADQL Enhancements Clive Page, AstroGrid Project University of Leicester, UK.

2003 May 24Clive Page Simple ADQL Enhancements Clive Page, AstroGrid Project University of Leicester, UK.
3 September 2004NVO Coordination Meeting1 Grid-Technologies NVO and the Grid Reagan W. Moore George Kremenek Leesa Brieger Ewa Deelman Roy Williams John.

3 September 2004NVO Coordination Meeting1 Grid-Technologies NVO and the Grid Reagan W. Moore George Kremenek Leesa Brieger Ewa Deelman Roy Williams John.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.

September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.

September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
VO Standards – Catalog Access Tamás Budavári Johns Hopkins University.

VO Standards – Catalog Access Tamás Budavári Johns Hopkins University.
Abstraction Layers Why do we need them? –Protection against change Where in the hourglass do we put them? –Computer Scientist perspective Expose low-level.

Abstraction Layers Why do we need them? –Protection against change Where in the hourglass do we put them? –Computer Scientist perspective Expose low-level.
Enterprise Java and Data Services Designing for Broadly Available Grid Data Access Services.

Enterprise Java and Data Services Designing for Broadly Available Grid Data Access Services.
OGSA-DAI Render Demo NeSC Review 18 March 2004. Description and Aims The OGSA-DAI Render demo is intended to demonstrate both the Process and Data aspects.

OGSA-DAI Render Demo NeSC Review 18 March Description and Aims The OGSA-DAI Render demo is intended to demonstrate both the Process and Data aspects.
Www.edikt.org Eldas 1.0 Enterprise Level Data Access Services Design Issues, Implementation and Future Development Davy Virdee.

Eldas 1.0 Enterprise Level Data Access Services Design Issues, Implementation and Future Development Davy Virdee.
E-Science Data Information and Knowledge Transformation Edikt : e-Science Data, Information and Knowledge Transformation NeSC Review, 30 September 2003.

E-Science Data Information and Knowledge Transformation Edikt : e-Science Data, Information and Knowledge Transformation NeSC Review, 30 September 2003.
A PPARC funded project The Grid Data Warehouse Description of prototype work in progress by AstroGrid. Access-Grid lecture to Universities of Leeds and.

A PPARC funded project The Grid Data Warehouse Description of prototype work in progress by AstroGrid. Access-Grid lecture to Universities of Leeds and.
VO-DAS Chenzhou CUI Chao LIU, Haijun TIAN, Yang YANG, etc National Astronomical Observatories, CAS.

VO-DAS Chenzhou CUI Chao LIU, Haijun TIAN, Yang YANG, etc National Astronomical Observatories, CAS.
Simple 3 layer model From Database to User Interface - Modules © Juhani Välimäki 2004.

Simple 3 layer model From Database to User Interface - Modules © Juhani Välimäki 2004.
AstroGrid Consortium Meeting PM Report. 26-27.06.2002AstroGrid Consortium Meeting Overview Activities Finance Recruitment Collaboration Phase B.

AstroGrid Consortium Meeting PM Report AstroGrid Consortium Meeting Overview Activities Finance Recruitment Collaboration Phase B.
What is MySQL? MySQL is a relational database management system (A relational database stores data in separate tables rather than putting all the data.

What is MySQL? MySQL is a relational database management system (A relational database stores data in separate tables rather than putting all the data.
A PPARC funded project AstroGrid Grid Data Warehouse Iteration 5 workgroup meeting.

A PPARC funded project AstroGrid Grid Data Warehouse Iteration 5 workgroup meeting.
G O B E Y O N D C O N V E N T I O N WORF: Developing DB2 UDB based Web Services on a Websphere Application Server Kris Van Thillo, ABIS Training & Consulting.

G O B E Y O N D C O N V E N T I O N WORF: Developing DB2 UDB based Web Services on a Websphere Application Server Kris Van Thillo, ABIS Training & Consulting.
Writing Perform Documents EPCC, University of Edinburgh Amy Krause ( Tom Sugden First International Summer.

Writing Perform Documents EPCC, University of Edinburgh Amy Krause ( Tom Sugden First International Summer.

Similar presentations

Ads by Google