Academic Technology Services The UCLA Grid Portal - Campus Grids and the UC Grid Joan Slottow and Prakashan Korambath Research Computing Technologies UCLA Academic Technology Services

Academic Technology Services https Uniform browser based interface Credential mapped to local ID Grid Appliance Head node Cluster II Head node Cluster I Credential mapped to local ID Grid Portal Grid Appliance ION Visualization ServerStorage Server UCLA Grid Architecture MyProxy Server

Academic Technology Services Single Campus Architecture Campus Grid Portal and CA A A C C Campus MyProxy Server Campus Storage Server A = Grid Appliance C = Cluster and Cluster Head Node Grid Portal accesses the storage server via GridFTP. No other access is provided.

Academic Technology Services Single CA is shared among all campuses UC Portal UCLA Portal UCSB Portal UCI Portal A A AA A A C C CC C C All appliances talk to both their campus portal and the UC portal UCLA MyProxy UCSB MyProxy UC Grid CA UC User Database UC MyProxy UC Register Service Storage Server Storage Server Storage Server UCI MyProxy Multi-Campus Architecture for the UC

Academic Technology Services Pools Resource Pool: –Clusters contribute cycles to the resource pool –Clusters contribute applications to a resource pool: –TeraGrid cycles purchased by the campus can easily be shared. Pool Users: –Every cluster user is also a pool user. –Many students and faculty members do not currently have login ids on any of the clusters. They can easily sign up to be pool-only users. –Each pool-only user is assigned a Storage Area. Pool use hides all the details Pool jobs –Target cluster selected by the UCLA Grid Portal. –Currently run applications only. A similar sharing of resources and applications can be applied across the entire UC.

Academic Technology Services User Types 2 Types of users: –Cluster User: Has a login ID on at least 1 cluster –Pool-Only User: Does not have a login ID on any cluster Both types of users have certificates/usernames to login to the Grid Portal

Academic Technology Services User Types vs. Portals Portal UserTypeUserType Campus PortalUC Portal Pool-only User Cluster User Can submit jobs to the campus pool Can submit jobs to the UC pool Can use those clusters on campus he/she can access Can submit jobs to the campus pool Best choice for those with access to clusters on different campuses -- can use all clusters, UC-wide, he/she can Access Can submit jobs to the UC pool

Academic Technology Services To Login to a Portal a User Needs: A certificate A gridsphere account on that Portal Additionally: A Cluster User must be added to the gridmap file on the appliance node of each of the clusters on which he/she has a login id A pool-only user needs to be assigned a storage area on the Grid Portal’s Storage Server.

Academic Technology Services Workflow -- For the UC Grid -- 1 Campus Grid Portal All authentication is done at the campus level Sends a message to campus grid admin User Authentication Form requesting Proposed username Unique? Requested username/password pending Usernames are unique across the entire UC UC Grid CA UC MyProxy UC Register Service UC User Database Sends a message to cluster admin Pool-only user Cluster user

Academic Technology Services Workflow -- For the UC Grid -- 2 Campus Grid Portal Authenticates Campus Grid Admin Creates GridSphere account (assigns storage area) Clicks Approve Sends request + Host Cert Campus MyProxy UC Register Client UC Grid CA UC MyProxy UC Register Service UC User Database Creates and signs user certificate Creates gridsphere account at UC Portal Pushes certificate to UC MyProxy Pushes certificate to campus MyProxy Requested username/password pending

Academic Technology Services

Pool Job Submission Submitting a pool job

Academic Technology Services How Pools Work Currently applications only A cluster that contributes to the pool must create: –Guest login id –Certificate for Grid Portal –Put guest in the gridmap file –Mechanism that allocates resources for pool jobs is up to the cluster administrator

Academic Technology Services Pool Job Submission Portal User Storage Area Find the best cluster Select a guest user Generate guest user proxy Stage input files to target cluster Database Status Change ? Update Pending Active Done Error Job submitted via GRAM Job Service and the guest user proxy On Storage Server or Originating Custer Guest User Storage Area A C Create a uniquely-named subdirectory Add Database entry for this job When notified by GRAM of a status change Update the database Target Cluster

Academic Technology Services Pool Job Output

Academic Technology Services Pool Job Output Retrieving Pool Job Output

Academic Technology Services TeraGrid UGP acts as a client to the TeraGrid. Their Globus Toolkit nodes are open to all of their users. They allowed us to get the CA Signing Policy and CA Certificate Public Key for each of their clusters. It only goes one way. User Proxy Certificate myproxy.ncsa.uiuc.edu UCLA Grid Portal Request for user proxy User proxy certificate returned because UCLA Grid Portal can authenticate DataStar at SDSC user proxy certificate UCLA pool jobs currently can run on two TeraGrid clusters.

Academic Technology Services Submitting Jobs to the TeraGrid Submitting a job to the TeraGrid

Academic Technology Services Grid Development Environment and Ajax Technologies Ajax Ajax -- Asynchronous JavaScript and XML Create web applications that are more interactive Only those parts of a page that change are updated GDE Create project, import files, save project Fully integrated editor interface GDE-generated Makefile or use pre-existing user Makefile Can for as many windows as you want. (Can be on different clusters) Tested in IE, Safari, Firefox Uses Zimbra Open AJAX Toolkit (BSD license) Compilations submitted to instantaneous queues (must run on the cluster, not on the appliance)

Academic Technology Services GDE Ajax Grid Development Environment GDE Project Directory

Academic Technology Services Visualization

Academic Technology Services Pools -- Now Single Pool –Every campus cluster user is also a pool user. –Pool-only users can only submit to the pool –Any cluster can contribute cycles to the pool It can choose which of its applications it will contribute to the pool When a pool job is submitted, the Portal decides which cluster will run it.

Academic Technology Services Futures -- Specialty Pools Clusters can form Specialty Pools –A user must be approved by the pool admin to –A specialty pool includes a list of applications –Grid Portal databases for: pools and pool users

Academic Technology Services UCLA UC Matlab Amber Q-Chem Mathematica Matlab Amber Q-Chem Gromacs Vasp Jaguar Amber Fluent UCLA UCSB TeraGrid UCI Mathematica Corsica Osiris baby b2 Jaguar * * * = CNSI Mathematica Gromacs * * * Joan is in the default pools Frank is in the default pools and the UCLA Physics Pool Prakashan is in the default and chemistry pools Futures -- Specialty Pools

Academic Technology Services Futures -- Redundancy and Failover Also called big IP It could be Apache or it could be hardware Farms out work to the portals.... MySQL Master Slave Disk May be a shared file system RAID5 or just like an appliance LB MyProxy RAID5 Load Balancer Load Balancer This is a feature of MySQL which automatically mirrors the database on the slaves. Portal LB

Academic Technology Services Workflow for Cluster User -- Now -- Single Campus Cluster Head node SSH authentication Sends a message to cluster admin Apply Page Approves and adds to the gridmap file Sends a message To grid admin Creates gridsphere account Sends a message To the user Request for password, create and sign certificate, push it to MyProxy User

Academic Technology Services Workflow for Pool-only User -- Now -- Single Campus ISIS used at UCLA link Apply Page User authenticated if he/she belongs to the UCLA community Sends a message To grid admin Sends a message To the user User can now log in User Purse (on MyProxy Server) Creates gridsphere account and assigns storage area on storage server

Academic Technology Services Workflow -- Add a Cluster on Another Campus -- 1 Other Campus Grid Portal Main Page Sends a message to other campus grid admin User Username Authentication UC Grid CA UC MyProxy UC Register Service UC User Database Sends a message to cluster admin GridSphere account not required

Academic Technology Services Workflow -- Add a cluster on Another Campus -- 2 Other Campus Grid Portal Authenticates Other Campus Grid Admin Creates GridSphere account Clicks Approve Sends request + Host Cert Other Campus MyProxy UC Register Client UC CA UC MyProxy UC Register Service UC User Database Pushes certificate to Other campus MyProxy

Academic Technology Services Portal Decides Where the Job will Run Amber Job Submitted Requirements? Time, Number of nodes Current Load Queue Length Clusters with Amber Can resources meet requirements? Select one

Academic Technology Services UCSB Batch Queue Prediction Algorithm UCSB Prediction Algorithm: Daniel Nurmi 75% of the jobs queued start in 317 seconds ~ 5 minutes 95% of the jobs queued start in 54,000 seconds ~1.5 hours Hoffman cluster Parallel Queue

Academic Technology Services Futures -- Job Status and the UC Grid Portal A A AA C1 C2 C1C2 Jobs DB Jobs DB Jobs DB Joan submits a job to: - UCLA C1 - UCLA C2 - UCLA pool From the UCLA Grid Portal Joan submits a job to: - UCLA C1 - UCI C1 - UC Pool From the UC Grid Portal UCLA Portal UCI Portal UC Portal

Academic Technology Services