1. Scaling NVO Services to the Teragrid Roy Williams Conrad Steenberg Craig Miller Matthew Graham Joe Jacob Julian Bunn

2. Desired Characteristics of NVO Services Service oriented architecture Services should be easily and quickly deployable and usable on workstations or supercomputers Services deployed, managed, and upgraded by their developers Service developers/deployers are trusted users Service developer acts as a broker between computing customer and computer center Service users authenticated with “graduated security” Easy to start, but great power is possible Asynchrony for compute intensive jobs Jobs submitted to batch queue Unique sessionID may be used to monitor job & return results From “clicking” to “scripting” Services may be accessed by clicking on a web page or with scripted client codes Authentication for web clicking comes from a certificate store or fat browser Scripted access requires a certificate (strong or weak) straight from the client Services as workflow components A service user may be another service (a computer, not a human!)

3. A “Graduated Security” Model Web form - anonymous access, small jobs Some science.... Get NVO weak certificate - access logged, but identity not verified More science.... Full TeraGrid account - browser access Big-iron computing.... Scripted access Power user Portal-Based

4. Traditional Grid Security client Show us your Certificate! I will do exactly what you want.

5. Graduated Security client May I have your Request and your Certificate?

6. This is a US driver’s licence. In the US it proves identity strongly. It is like a strong certificate. This is a loyalty card where I buy food. (You can put a false address on the application.) It is like a weak certificate. This is a $50 gift card at a bookstore. It does not prove my identity in any way. It is like an anonymous certificate. Certificates The Virtual Observatory as a Virtual Organization

7. "nesssi_strong_cert_max_time" : 216000, "nesssi_weak_cert_max_time" : 3600, "nesssi_anon_max_time" : 900, "nesssi_anon_user" : ”nvo", "nesssi_weak_user" : "nvo", Graduated security No certificate gets 15 CPU-minutes from community account Just switch on Javascript Weak certificate gets 1 CPU-hour from community account In exchange for registering name/email Strong (gridmapped) certificate gets infinity from own account Get this one from TeraGrid HQ

8. service implementation web forms python API graduated security Certificates multiple browsers certificate chains root certificates proxy certificates proxy certificate chains 2nd level proxy chains secure https redirection teragrid security police caltech security police NCSA security police chown directory ownership NFS root-squashing pubcookie

9. Three Interfaces Commandline with Python & Java Cert or proxy in wacko place like.globus or /tmp/u509 Fat Browser https: and browser managed PKCS12 certs Thin Browser Web Proxy works dynamically with cert authority

10. Commandline Portal clientnesssi node Teragrid cluster certificate policies queue select user account sandbox storage XML-RPC proxy open http build proxy Certificate Authority get certificate

11. Fat Browser Portal Browsernesssi node Teragrid cluster certificate policies queue select user account sandbox storage JSON-RPC certificate open http Certificate Authority load certificate

12. Web Portal client certificate services nesssi web portal nesssi node web form SOAP http queue fetch proxy select user account sandbox storage open http certificate policies

13. Exercise: Running a Nesssi Service see http://us-vo.org/nesssihttp://us-vo.org/nesssi

14. SessionID and Sandbox Identify which job we are talking about 32 character hex string eg cb28d0753a7fec9a485981f741d425ec Used to monitor a running job sessionID = remoteserver.cutout.init() msg = remoteserver.cutout.monitor(sessionID) Used to form URL where results appear, eg http://dtf-test1.sdsc.teragrid.org:8080 /clarens/shell/cb/cb28d0753a7fec9a485981f741d425ec/cutouts/index.htm If you lose the sessionID, you lose your job

15. DPOSS Mosaic Service nesssiServer=nesssi.client('https://envoy.c acr.caltech.edu:8443/clarens/',debug=0) mosaic_loc = "-ra 49.1 -dec 60.1 -rawidth 0.5 -decwidth 0.5 -filt f -bgcorr 0" session = nesssiServer.dpossMosaic.mosaic(mosaic_loc) print "Your session ID is %s." % session msg = dbsvr.dpossMosaic.monitor(session) print msg Repeat the monitoring

17. nesssiServer. dpossMosaic.mosaic ( “-ra 49.1 -dec 60.1 -rawidth 0.5 -decwidth 0.5 -filt f -bgcorr 0”)

18. Cutout Service nesssiServer=nesssi.client('https://envoy.cacr.caltech.edu: 8443/clarens/',debug=0) sessionID = nesssiServer.cutout.init() print "Session id is ", sessionID # Upload locations file nesssiServer.upload_file(“inputfile.xml”, ”inputfile.xml”) # Arguments for service, surveys to use and cutout size args = "-surveys PQ:gr,PQ:gi,PQ:z1,PQ:z2,SDSS:r,SDSS:i,SDSS:z,2MASS:k,2MASS:h " args += "-size 64" # Run service nesssiServer.cutout.run(sessionID, args)

19. Cutout Monitoring

20. cutouts from Palomar-Quest, SDSS, 2MASS of sources from Veron quasar catalog

23. Synoptic Coaddition service Palomar-Quest Survey Coverage map Max=18

24. Making a Service Developer builds script Keyword-value pairs on command line [+uploaded files] Sandbox location given on cmdline -- all files staged there Should make index.htm in sandbox for progress Make Nesssi connector for init(), upload(), run(), monitor() Nesssi admin installs your service Interview first Symlink to code Code is cached, restart server after edit Developer gets right to restart server (running jobs not affected) Service instantiations farmed out to cluster with PBS

25. Server side code Application example: dposs.py -dir sandbox \ -ra 123 -dec 22.7 \ -rawidth 0.4 -decwidth 0.4 \ -filt j -bgcorr 1 It should: (1) Use keyword-value arguments and uploaded files (2) Read/Write results in the given sandbox directory (3) Write a progress file in sandbox/index.htm (4) Estimate limits for anon/weak/strong certs Service code will be symlinked from server code directory Requires sudo server restart to see the service

26. Client-side Javascript <input type="button" onclick="connect_nesssi('dposs')" name="Connect" value="Connect to Nesssi"> Service name Expect to run remote services called: dposs.init(), dposs.run(), dposs.monitor()

27. Client-side Javascript <input name="ra" value="202.4682” Etc… function getparams() { var params = "-ra " + document.Parameters.ra.value + " " + "-rawidth " + document.Parameters.rawidth.value + " " + "-dec " + document.Parameters.dec.value + " " + "-decwidth " + document.Parameters.decwidth.value + " " + "-filt " + filt + " " + "-bgcorr " + bgCorr; return params; } The form for the user Developer converts the form to a string

28. Nesssi Assets Graduated security Anonymous, Registered, Known Multiple interfaces Fat browser, Web proxy, Scripted Multiple implementations cacr.caltech.edu and sdsc.teragrid.org Some useful services Hyperatlas mosaic, Cutouts, Synoptic coaddition Teragrid acceptance of security model Server runs a job as somebody else Anonymous access to TeraGrid!!