A PPARC funded project AstroGrid’s Common Execution Architecture Guy Rixon, reporting on behalf of Paul Harrison and the other AstroGrid developers SC4DEVO-1, Pasadena, July 2004

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 2 AstroGrid UK national virtual-observatory project See cover slide for participants and funding Tasked to produce a VO toolkit to be reused by service providers Collective author of a SOA using web services Common Execution Architecture: A major part of AstroGrid architecture Part of the service-oriented architecture A way of making and using web services Some important applications to data mining

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 3 SOA for data mining How can we carve up the distributed application into services? And where do we put the application logic?

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 4 Distributed app: conservative Desktop Application Data-centre 1 Data source Data-centre 2 Data transform Application logic in here

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 5 Distributed app: browser-based Desktop Browser Data-centre 1 Data source Data-centre 2 Data transform Application logic in here

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 6 Distributed app: intragrid Desktop Browser/app Data-centre 1 Data source Data-centre 2 Data transform Compute grid Node Some application logic in here; Rest still in client or in user

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 7 Distributed app: scripted Desktop Browser/app Data source Data-centre 2 Data transform Data centre 3 Data transform Data-centre 1 Portal Workflow engine Script App logic in here

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 8 Distributed app: embedded Desktop Browser Data source Data-centre 2 Data transform Data mine Application Data-centre 1 Portal Workflow engine Script Most app logic here Little app logic here

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 9 Distributed app: data grid Desktop Browser/app Data source Data-centre 2 Data transform Data centre 3 Data transform Data-centre 1 Portal Workflow engine Script

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 10 Issues to be addressed Boring, technical: How to write apps as w/s? How to write w/s clients? How to connect apps to storage? How to register functions? Interesting, scientific: Who has useful algorithms? How to match algorithms to data? Best place to run computations? How to share code? Provenance? How to share kudos? Is more science possible?

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 11 AstroGrid architecture and products AstroGrid bits and pieces let techies solve the tech problems so that scientists can concentrate on the science issues. AstroGrid supports: Browser-based app Scripted app Data grid Embedded app (pre-installed exes) Common Execution Architecture lets all parts but exe for embedded app be reusable code.

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 12 Common Execution Architecture (1) Desktop Browser/app Data source Data-centre 2 Data transform Data centre 3 Data transform Data-centre 1 Portal Workflow engine Script Web service (CEC) Application (local exe) MySpace Data mine “CEC” = Common Execution Connector CEC configuration

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 13 Common Execution Architecture (2) Desktop Browser/app Data source Data-centre 2 Data transform Data centre 3 Data transform Data-centre 1 Portal Workflow engine Script Workflow executor (JES) MySpace Page presentation Workflow definition Portal Scripts stored here, as files

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 14 Common execution architecture (3) Common language throughout system allows s/w reuse.

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 15 How to write an app using CEA? Don’t write a web service! Write a command-line exe Configure a CEC to run your app: I.e. describe parameters in the CEA language Or: get your app configured on someone else’s CEC

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 16 How to publish an app using CEA? Take the that informs the CEC. Wrap it in a of sub-type CeaApplicationType. Publish the in IVO registry.

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 17 How to call an app-service using CEA? Look it up in the resource registry. Get the from its. Give the to a CEA delegate => define SOAP call. Add parameters from UI according to. Bake in pre-heated SOAP HTTP 1.1.

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 18 CEA language example Detection Image The image that is used to detect sources. Basic position, shape and size information is derived from this image Measurement Image The Image that is used to measure photometric parameters …

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 19 CEA language schema (1)

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 20 CEA language schema (2)

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 21 CEA facilities so far PAL CEC HyperZ CEC Other service MySpace JES CEA DelegateOther Delegate SIAP CEC

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 22 Future evolution Add facilities to CEC: Asynchronicity Security More parameter types New facility: field-programmable CEC (PCEC)

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 23 PCEC (1): local code libraries Client PCEC Data mine Code library Load exe Select exe

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July Slide 24 PCEC (2): distributed code-library Client PCEC Data mine Code library Download exe Select exe VOSpace Sandbox Run exe Upload exe