Presentation is loading. Please wait.

Presentation is loading. Please wait.

Asynchronous services from NVO Roy Williams Conrad Steenberg Craig Miller Matthew Graham Joe Jacob Julian Bunn.

Similar presentations


Presentation on theme: "Asynchronous services from NVO Roy Williams Conrad Steenberg Craig Miller Matthew Graham Joe Jacob Julian Bunn."— Presentation transcript:

1 Asynchronous services from NVO Roy Williams Conrad Steenberg Craig Miller Matthew Graham Joe Jacob Julian Bunn

2 Why Nesssi? Graduated security Anonymous, Registered, Known Multiple interfaces Fat browser, Web proxy, Scripted Astrogrid pattern Nesssi services can easily be Astrogrid services Can be discovered by VO registry Can be used as part of Astrogrid workflow Multiple implementations cacr.caltech.edu and sdsc.teragrid.org Some useful services Mosaic, Cutouts, Synoptic coaddition Mashup of VO SIAP services Teragrid acceptance of security model Server runs a job as somebody else Anonymous access to TeraGrid!!

3 A “Graduated Security” Model Web form - anonymous access, small jobs Some science.... Get NVO weak certificate - access logged, but identity not verified More science.... Full TeraGrid account - browser access Big-iron computing.... Scripted access Power user Portal-Based Request and Certificate are judged *together*

4 "nesssi_strong_cert_max_time" : 216000, "nesssi_weak_cert_max_time" : 3600, "nesssi_anon_max_time" : 900, "nesssi_anon_user" : ”nvo", "nesssi_weak_user" : "nvo", Graduated security No certificate gets 15 CPU-minutes from community account Just switch on Javascript or connect to portal Weak certificate gets 1 CPU-hour from community account In exchange for registering name/email Can do scripted access Strong (gridmapped) certificate gets infinity from own account Get this one from TeraGrid HQ

5 Multiple Interfaces Fat Browser https: and browser managed PKCS12 certs Thin Browser Web Proxy works dynamically with cert authority Commandline with Python No Globus install needed! Astrogrid Application Description allows publish/discover/workflow

6 Fat Browser Portal Browsernesssi node Teragrid cluster certificate policies queue select user account sandbox storage JSON-RPC certificate open http Certificate Authority load certificate

7 Web Portal client certificate services nesssi web portal nesssi node web form SOAP http queue fetch proxy select user account sandbox storage open http certificate policies

8 Commandline Portal clientnesssi node Teragrid cluster certificate policies queue select user account sandbox storage XML-RPC proxy open http build proxy Certificate Authority get certificate

9 Astrogrid workflow AG server node Grid cluster queue sandbox storage SOAP Request/response workflow

10 SessionID and Sandbox Identify which job we are talking about 32 character hex string eg cb28d0753a7fec9a485981f741d425ec Used to monitor a running job sessionID = remoteserver.cutout.init() msg = remoteserver.cutout.monitor(sessionID) Used to form URL where results appear, eg http://dtf-test1.sdsc.teragrid.org:8080 /clarens/shell/cb/cb28d0753a7fec9a485981f741d425ec/cutouts/index.htm If you lose the sessionID, you lose your job

11 Mosaic Service nesssiServer=nesssi.client('https://envoy.c acr.caltech.edu:8443/clarens/',debug=0) mosaic_loc = "-ra 49.1 -dec 60.1 -rawidth 0.5 -decwidth 0.5 -filt f -bgcorr 0" session = nesssiServer.dpossMosaic.mosaic(mosaic_loc) print "Your session ID is %s." % session msg = dbsvr.dpossMosaic.monitor(session) print msg Repeat the monitoring

12

13 Cutout Service nesssiServer=nesssi.client('https://envoy.cacr.caltech.edu: 8443/clarens/',debug=0) sessionID = nesssiServer.cutout.init() print "Session id is ", sessionID # Upload locations file nesssiServer.upload_file(“inputfile.xml”, ”inputfile.xml”) # Arguments for service, surveys to use and cutout size args = "-surveys PQ:gr,PQ:gi,PQ:z1,PQ:z2,SDSS:r,SDSS:i,SDSS:z,2MASS:k,2MASS:h " args += "-size 64" # Run service nesssiServer.cutout.run(sessionID, args)

14 Cutout Monitoring

15 cutouts from Palomar-Quest, SDSS, 2MASS of sources from Veron quasar catalog

16 Synoptic Coaddition service Palomar-Quest Survey Coverage map Max=18

17 Making a Service Developer builds script Keyword-value pairs on command line [+uploaded files] Sandbox location given on cmdline -- all files staged there Make index.htm in sandbox for progress Make Nesssi connector for init(), upload(), run(), monitor() Make Astrogrid service description Nesssi admin installs your service Interview first Symlink to code Code is cached, restart server after edit Developer gets right to restart server (running jobs not affected) Service instantiations farmed out to cluster with PBS

18 Server side code Application example: mosaic.py -sandbox my-output-dir \ -ra 123 -dec 22.7 \ -rawidth 0.4 -decwidth 0.4 \ -survey SDSS:i -bgcorr 1 It should: (1) Use keyword-value arguments and uploaded files (2) Read/Write results in the given sandbox directory (3) Write a progress file in sandbox/index.htm (4) Estimate limits for anon/weak/strong certs Astrogrid services are also driven by keyword-value pairs -- this is why every Nesssi service can be an Astrogrid service


Download ppt "Asynchronous services from NVO Roy Williams Conrad Steenberg Craig Miller Matthew Graham Joe Jacob Julian Bunn."

Similar presentations


Ads by Google