1. PRAGMA Application (GridFMO) on OSG/FermiGrid Neha Sharma (on behalf of FermiGrid group) Fermilab Work supported by the U.S. Department of Energy under contract No. DE-AC02-07CH11359 PRAGMA 13 Workshop - September 23-25 2007

2. Table of Contents Goal/People Involved GridFMO OSG FermiGrid Meeting the Requirements Milestones Monitoring Plots Software Documentation PRAGMA 13 Workshop - September 23-25 2007 2

3. Goal Successful run of PRAGMA application (GridFMO) on OSG/FermiGrid resources. Demonstration of Interoperability for job execution between FermiGrid/OSG and PRAGMA infrastructures. Provide feedback to technology directions of FermiGrid/OSG to better support more general interoperation in the future PRAGMA 13 Workshop - September 23-25 2007 3

4. People Involved Cindy Zheng - PRAGMA coordinator Tsutomu Ikegami & Yoshio Tanaka- PRAGMA application leads Neha Sharma - OSG/FermiGrid coordinator Steve Timm - FermiGrid Systems Administrator PRAGMA 13 Workshop - September 23-25 2007 4

5. GridFMO GridFMO is a Grid version of the Fragment Molecular Orbital (FMO) calculation. FMO method enables a first- principle calculation of macromolecules such as proteins. In the FMO method, the electronic state of the whole molecule is calculated by splitting it into small fragments. Electronic state calculations can be performed independently (in parallel) and thus are a good candidate for grid. The results of these calculations are used to draw potential energy curves of proteins. PRAGMA 13 Workshop - September 23-25 2007 5 http://goc.pragma-grid.net/applications/FMO/FMO.html

6. Open Science Grid OSG provides a common, shared, distributed infrastructure where resources (retained under control of the owners) are accessible to scientific applications (including services needed by the research group itself). The owners of the resources provide both guaranteed and opportunistic use of available compute cycles and managed storage. OSG interoperates with campus, regional, national and international cyber-infrastructures towards a worldwide distributed computing network for scientists, researchers, educators, and students. OSG Consortium currently has ~100 institutions and ~36 research and IT organizations. www.opensciencegrid.org PRAGMA 13 Workshop - September 23-25 2007 6

7. FermiGrid FermiGrid is an example of a common, shared campus infrastructure. FermiGrid allows VOs to submit jobs and data through a centralized interface to be dispatched to the available resources (these resources are not all directly accessible from outside). FermiGrid provides central services (authentication, authorization & monitoring) for all the local resources. http://fermigrid.fnal.gov PRAGMA 13 Workshop - September 23-25 2007 7

8. FermiGrid Resources PRAGMA 13 Workshop - September 23-25 2007 8 Metafacility consists of various compute and storage resources at Fermilab Compute resources CDF CMS Dzero General Purpose Grid cluster Total ~ 9000 slots ~ 2100 worker nodes Storage resources Fermilab public storage- 7 TB Blue Arc (NAS) - 14.6 TB

9. VOMS Server BlueArc Site Wide Gateway FermiGrid - Current Architecture CMS WC2 CDF OSG1 CDF OSG2 D0 CAB1 GP Farm SAZ Server GUMS Server Step 1 - user issues voms-proxy-init user receives voms signed credentials Step 2 – user submits their grid job via globus-job-run, globus-job-submit, or condor-g Step 4 –Gateway requests GUMS Mapping based on VO & Role Step 3 – Gateway checks against Site Authorization Service clusters send ClassAds via CEMon to the site wide gateway Step 5 - Grid job is forwarded to target cluster Periodic Synchronization D0 CAB2 Exterior Interior CMS WC1 PRAGMA 13 Workshop - September 23-25 2007 9 * Courtesy of Keith Chadwick (FermiGrid)

10. Meeting the requirements… VO membershipIt was decided that jobs would run under VO=GIN and group=pragma. GIN - Grid Interoperability Now GridFMO: Application needed to be tested on environment similar to execute nodes Fermilab off-site ssh based login account was created Fermilab/Open Science Enclave: No non-kerberos logins by grid users. Intra-node parallel execution model was used. This eliminated the need for rsh/ssh login. Another application model was based on inter-node communication and required rsh/ssh based logins. GridFMO: Application required certain number of nodes to be available simultaneously. Globus RSL parameter condorsubmit was used to request nodes which were idle at time of submission. PRAGMA 13 Workshop - September 23-25 2007 10

11. Milestones Deadline: March 5'2007 December 2006 Identified contacts January 2007 Identified application (purpose and requirements), tools etc. February 2007 Fermilab Offsite user account (setup on FermiGrid) DN added to GUMS database Simple globus-job-run jobs successful Test GridFMO job successful March 2007 - June 2007 Application run finished successfully(4-6 days@7calculations) PRAGMA 13 Workshop - September 23-25 2007 11

12. Unique VOs on FermiGrid by Month (stacked) * Plot - Courtesy of Keith Chadwick (FermiGrid) PRAGMA 13 Workshop - September 23-25 2007 12

13. Unique VOs on OSG by Month (stacked) * Plot - Courtesy of Keith Chadwick (FermiGrid) PRAGMA 13 Workshop - September 23-25 2007 13

14. Software The OSG Virtual Data Toolkit (VDT) provides an integrated distribution of the underlying software components (including Condor and Globus). FermiGrid uses the VDT internally and contributes to its testing and evolution. Some components are in common with PRAGMA sites and needs. Is this an area for further collaboration? PRAGMA 13 Workshop - September 23-25 2007 14 http://vdt.cs.wisc.edu/

15. Documentation http://goc.pragma-grid.net/wiki/index.php/OSG- PRAGMA_Grid_Interoperation_Experiments http://goc.pragma-grid.net/wiki/index.php/OSG- PRAGMA_Grid_Interoperation_Experiments http://fermigrid.fnal.gov/gin/pragma/osg-pragma- interoperability.html http://fermigrid.fnal.gov/gin/pragma/osg-pragma- interoperability.html PRAGMA 13 Workshop - September 23-25 2007 15

16. This OSG-PRAGMA interoperation experiment was a success and is a stepping stone towards achieving higher levels of interoperability ! OSG has also taken one of its applications (GISOLVE) and run this on a PRAGMA site (rocks-52.sdsc.edu) We are interested in trying other applications between the two infrastructures. Thank you Finally... PRAGMA 13 Workshop - September 23-25 2007 16