Presentation is loading. Please wait.

Presentation is loading. Please wait.

Science Gateways Objectives aka Nancy’s Brave New Gateway World Quarterly Meeting, December 6-7, 2007.

Similar presentations


Presentation on theme: "Science Gateways Objectives aka Nancy’s Brave New Gateway World Quarterly Meeting, December 6-7, 2007."— Presentation transcript:

1 Science Gateways Objectives aka Nancy’s Brave New Gateway World Quarterly Meeting, December 6-7, 2007

2 Gateway Objectives for PY4 and 5 TeraGrid integration will be straightforward for new and existing gateway developers There will be a set of easy to discover general services provided by and for Gateways The targeted support program will be well- organized We will be able to routinely count end gateway users, who will total 25% of total TeraGrid users There will be a funded cross-directorate gateway program at the NSF Quarterly Meeting, December 6-7, 2007

3 TeraGrid integration will be straightforward for new and existing gateway developers Clear documentation, including step by step integration instructions for gateways –These instructions will include all gateway requirements. Deployment of build your own gateway capability Tools necessary for tasks like accounting and authentication Quarterly Meeting, December 6-7, 2007

4 There will be a set of easy to discover general services provided by and for Gateways There will be a published list of general web services for Gateways Gateways will be able to easily make their own services available to others Quarterly Meeting, December 6-7, 2007

5 The targeted support program will be well- organized The request process will be clear –PIs will understand the status of their requests, It will be clear to staff members what they are working on and when –We’ll be able to more easily transition amongst projects Lessons learned will be included in general gateway documentation/case studies We will work with at least 10 new projects in PYs 4 and 5 We will seek out projects benefiting underrepresented groups Quarterly Meeting, December 6-7, 2007

6 We will be able to routinely count end gateway users, who will total 25% of total TeraGrid users A unique identifier for each end gateway user per community account must exist in TGCDB Gateways will need to transmit and TGCDB will need to receive this additional identifier through any job submission mechanism Attribute-based authentication in production and easy to use Quarterly Meeting, December 6-7, 2007

7 There will be a funded cross-directorate gateway program at the NSF Recognizes the importance of gateways as infrastructure necessary to tackle the most compelling science problems Funds them in a sustainable, metrics-driven way Quarterly Meeting, December 6-7, 2007

8 Gateway Security Summit Meeting Proposed in San Diego for Jan/Feb The goal is to define once and for all –How gateways will use community accounts Shell access for developers? –How sites will secure these accounts Will developers need to make help desk requests for any writes to community account directory? The goal is not to have all sites to agree on how accounts will be secured –But we do need to define how all sites will secure these accounts and advertise this to users –Many examples of the process breaking today Quarterly Meeting, December 6-7, 2007

9 Recent Example of Problem to Solve National Biomedical Computation Resource Sent: Wednesday, October 31, 2007 5:57 PM I did some login and GSI authentication test for the "nbcruser" on Teragrid. Some sites don't work, can you help me to figure this out? First, I can get the proxy from myproxy.teragrid.org. SDSC: Everything works. NCSA: At the early of this month, I could login NCSA clusters from laptop and NBCR machines, and GSI authentication was also successfully. For now, I can not login NCSA with the passwd, authentication also failed, but I still can login NCSA from SDSC teragrid machine, and the authentication from SDSC to NCSA is successful. PSC: Login and authentication never work. Purdur Univ: The passwd of "nbcruser" isn't provided on the paper, and GSI authentication failed. TACC: I can login to the cluster, but the provided username is "tg459196", not the uniform "nbcruser"myproxy.teragrid.org Quarterly Meeting, December 6-7, 2007

10 Why a Face to Face Meeting? Considerable shuttle diplomacy between security- wg and gateways via Jim Marsteller and I The time has come for a face to face meeting to move this forward Critical Issues –Carefully define usage models –Carefully assess risk –Thoughtfully restrict accounts on par with risks Do not want a major gateway security incident Severe restrictions may have a significant impact on the gateway program Gateways reduce the impact of thousands of end user laptops, but may increase other risks Quarterly Meeting, December 6-7, 2007

11 TeraGrid is a service organization Must keep in mind what we ask of users Write allocation proposal –Follow our instructions, meet our timelines –Justification, paper listings, renewals each year Once successful PIs must –List gateway on public page URL and description –Request a community account Script locations, anticipated run sizes, anticipated data needs –Request developer accounts –Possibly request community software area (CSA) –Need to make sure community account group membership does not intersect with CSA group membership Quarterly Meeting, December 6-7, 2007

12 Now PI and developers are set up, what next? Integration into their own fully developed gateway –GRAM job submissions –Gridftp Identify striped and non-striped servers –Accounting GRAM audit Report to us quarterly on number of end users using gateways Future attributed-based authentication requirements –Credential management –May not be able to directly access files in community account We need to make sure the TeraGrid experience is worth this level of effort! Quarterly Meeting, December 6-7, 2007

13 Today’s Presentation Q1FY08 –Progress on top five goals –Activities Gateway Highlights Q2FY08 –Top five goals Quarterly Meeting, December 6-7, 2007

14 Q1FY08 (Oct-Dec) Top 5 1.Move dangerously close to routine, production use of TG 2.GRAM and gridFTP monitoring 3.Conduct gateway survey 4.Turn on community account portal 5.Have fun!

15 Q1FY08 Gateway Activities September –Presentation from Katherine Lawrence on TeraGrid Future project and associated June gateways workshop –EOT supplement activities –Jim Basney conducts certificate survey of gateways October –Discussion of log file uploads for security-wg –Resource selection via web services Initial work by Steve Mock, expanded by Stu Martin and Suresh Marru –Scientific and Scholarly Workflow Cyberinfrastructure workshop Dennis Gannon, Suresh Marru –Discussion with gateways about how to view WBS tasks, work with John Gerber on clear presentations November –GCE07 at SC07 Program committee members, numerous paper presentations –SC07 Education Program –Completed “community account central” portal for security-wg –Intensive LEAD debugging calls initiated –Gateway survey finalized, now need to conduct it! December Quarterly Meeting, December 6-7, 2007

16 Community Account Portal Completed! Designed for security-wg Password-protected internal site Type in community account username and bring up associated info Write permission for security staffers to add info about how and when these accounts have been secured at each site Repository for Gateway logs –Automatic mechanism and location for gateways to send logs. –Notification when logs are stale Perhaps recent usage data Quarterly Meeting, December 6-7, 2007

17 Thank You Quarterly Meeting, December 6-7, 2007

18 Quarterly Meeting SPRUCE Oct-Dec 07 Accomplishments Accomplished Milestones - –Prototype integration into HARC to support advance reservations as a policy –Enhanced site specific usage documentation –Integrated into SCOOP and HemeLB applications, demoed at SC07 –WS-GRAM extension testing –Ongoing tests for SPRUCE Advisor to predict ‘best’ resource –Basic infrastructure for on-demand bandwidth tokens via VLANS Science Impact –This quarter’s work enabled integration of some promising applications into the system. We had successful demonstrations of these applications at SC07. We also focused on making the documentation more user friendly. The prototypes for HARC and bandwidth token integration provide interesting avenues to explore in the coming days.

19 Quarterly Meeting SPRUCE Jan-Mar 08 Plans Road Map –Focus on movement into production as a CTSS component –LEAD real time spring runs for 2008 –Work on use cases for SCOOP and GENIUS HemeLB applications. –Production WS-GRAM support on all SPRUCE resources –Continue work on collaboration efforts with HARC and Condor –INCA application warm standby work Science Impact –The main focus of this quarter would be the requirements to move SPRUCE into production on TG resources. Apart from this we will continue to build up our collaboration efforts with other software as well as applications mentioned above. By the end of this quarter we hope to have a solid user base as well as a well defined plan for going into production.

20 Quarterly Meeting Science Gateway Credential Management Q1 Accomplishments –Surveyed current practice and identified security issues: teragridforum.org/mediawiki/index.php?title=Science_Gateway_Credential_Management Identified gateway type (portal, desktop, bridge) and credential type (individual or community) for each active science gateway Documented security issues with community accounts, community credentials, and desktop gateways Discussed issues and needs with Gateways and Security WGs Identified future work (documentation, policies, recommendations, audits) Q2 Plans –Develop best practice documentation with sample software Managing restricted community accounts Keeping CA certificates and CRLs up-to-date Authenticating users Tracking gateway usage Q3-Q4 Plans –Develop a security architecture for desktop gateways (i.e., GridChem and CSE-Online) –Provide credential management software for desktop gateways

21 Quarterly Meeting Attribute-based Access for Science Gateways Q1 Accomplishments –Led discussions with WGs on technical plans and policies: teragridforum.org/mediawiki/index.php?title=AAA_Testbed:Simple_Science_Gateway_with_Attribut es –Demonstrated attribute-based access from SimpleGrid portal to WS-GRAM node at NCSA Documentation at mywiki.ncsa.uiuc.edu/wiki/Modify_SimpleGrid –Presented “A Grid Authorization Model for Science Gateways” at GCE07 http://www.collab-ogce.org/gce07/index.php/Program Q2 Plans –Work with BioPortal Gateway developers on GridShib deployment –Develop documentation for deploying GridShib software at gateways and RPs Generalize documentation from SimpleGrid NCSA demonstration For tracking community user attributes (username, IP address, email address, and authentication timestamp) Q3-Q4 Plans –Develop mechanism to capture community user attributes in TGCDB –Explore integration with VOMS-based grids (such as OSG)

22 Quarterly Meeting GISolve – TeraGrid GIScience Gateway Progress Made During Q1FY08 (Oct. – Dec.) and Plans for Q2FY08 Shaowen Wang University of Illinois at Urbana- Champaign

23 Quarterly Meeting Q1FY08 Accomplishments Highlights Background –Geographic Information Science (GIScience), an interdisciplinary field involving geography and other social sciences, computer science, geodesy, information sciences, and statistics to study generic issues about the development and use of geographic information systems (GIS) technologies. Milestones –Regular number of users: 65 –Wrote the following paper based on the SimpleGrid toolkit experience Shaowen Wang, Yan Liu, Nancy Wilkins-Diehr, Stuart Martin, “SimpleGrid Toolkit: Enabling Efficient Learning and Development of TeraGrid Science Gateway.” Grid Computing Environments workshop at SC|07 –Education The following graduate and undergraduate courses at The University of Iowa is using GISolve –Bayesian Statistics (27 students) –Computing in Statistics (18 students) Impact on science –Produced the following research publications Wang, S, and Zhu, X-G. 2008. “Coupling Cyberinfrastructure and GIS to Empower Ecological and Environmental Research.” Forthcoming in BioScience, February, 2008 Wang, S., Cowles, M. K., and Armstrong, M. P. 2008. “Grid Computing of Spatial Statistics: Using the TeraGrid for Gi*(d) Analysis.” Concurrency and Computation: Practice and Experience, accepted Wang, S., and Armstrong, M. P. 2008. “A Theoretical Approach to the Use of Cyberinfrastructure in Geographic Analysis.” International Journal of Geographical Information Science, Accepted Yan, J., Cowles, M. K., Wang, S., and Armstrong, M. P. 2007. “Parallelizing MCMC for Bayesian Spatiotemporal Geostatistical Models.” Statistics and Computing, 17 (4): 323-335

24 Quarterly Meeting Q2FY08 Plans Milestones –Develop a new visualization service to support analysis steering –Explore new applications Land use and management for environmental sustainability Sustainability of Biomass-based energy –Continue to develop and support the SimpleGrid toolkit Describe impact on science GISolve couples cyberinfrastructure and GIS to empower ecological and environmental research. GISolve represents next- generation advanced Web-GIS that allows a large number of users to collaborate and share the workflows of geographic analyses and multi-scale modeling. A Bioenergy Data Portal in GISolve

25 Quarterly Meeting Caltech TeraGrid Science Gateways Status October-December 2007 Plans January-March 2008 Julian Bunn, Matthew Graham, John McCorquodale, Conrad Steenberg, Roy Williams

26 Quarterly Meeting Caltech TeraGrid Science Gateways Notable Events – Our gateway was moved from dtf-test1 to tg-c128 at SDSC – 18 different IP addresses/users accessed the Gateways in this quarter – Change provoked new review of security and new rules we had to follow for deployment, that continued into this quarter. A considerable sink of effort. – Concentrated effort to port Clarens server and services to new platform – Personnel changes: Conrad Steenberg moved to different department at Caltech, Nov 1. His TeraGrid GW tasks taken on by John McCorquodale (McQ). – NVO portal tools: smooth transitions from immediate job to batch job to TeraGrid job – NVO/HEP TeraGrid Gateways work presented at “Computing in High Energy Physics” (CHEP), Victoria, BC in September: www.chep2007.com – SuperComputing 07 : Demonstrated ROOTlets using local cluster of 32 nodes at Reno. Cluster used to demonstrate storage-storage transfers in WAN between Reno and global LHC sites. Aggregate I/O rates to node rack was 90 Gbps for single 50GB files over 7 10Gbit waves. More details: supercomputing.caltech.edu Plans – NVO gateways funding scheduled to terminate: no further development  – Initial discussion/exploration of a possible CIG gateway –Tests of ROOTlets with SDSC TeraGrid batch queues – McQ ramping up skills – Identify hard-core LHC analysis for expanded ROOTlet demo – Investigate deployment of ROOTlets on Indiana TeraGrid node(s)

27 Q4CY07 Updates – Dec. 6, 2007 nanoHUB Updates CY07 Q4 Accomplishments –NEMO_3D has been ported to BigRed, available to nanohub users now. –Abinit has been added at Argonne with anticipated testing at SDSC. We build our own copy so that we could have the version desired by the users at Norfolk State University. –A new application, "Multi-Gate Nanowire FET“, has been modified to send all of its jobs to a TG site. –Job submission interface is in place for both GUI driven and command line driven applications. Provisions have been added to designate use of remotely staged executables. This will come into play for adding access to NWChem, CPMD, LAMMPS, NAMD, Gaussian, and other applications. –Site selection process has been modified to test a narrower more stable set of sites for availability. The degree to which a site is stable is currently judged manually based on observation of the availability test results. TeraGrid jobs like all nanoHUB jobs are monitored and statistics are collected. –The present capacity level will allow for the parametric study of nano-electronic devices thereby advancing understanding of such devices. In Q4 the bringing online of other applications will offer similar capabilities in other fields of study. With more than 5,900 simulation users and 226,000 jobs, nanoHUB is expanding quickly. The increased user base demands that more compute intensive applications be made available. Use of TeraGrid resources will allow nanoHUB to service the increased demand. –Internal documentation has been written describing the job submission process. These will be made available to TG gateways community.

28 Q4CY07 Updates – Dec. 6, 2007 2007 Usage statistics 2007 Usage statistics ( for TG-enabled nanoHUB tools) MetricCY07 Q1Q2Q3Q4 (Oct,Nov) Number of unique nanoHUB users using the TG-enabled tools 289335256230 Total number of jobs2,1115,417281913,299 Hours total (wallclock)12,79211,880602415,096 Nanowire - Simulate electronic properties of silicon nanowire transistors. Nanofet - Simulates quantum ballistic transport properties in two-dimensional MOSFET devices Cntfet - Simulates ballistic transport properties in 3D Carbon NanoTube Field Effect Transistor (CNTFET) devices

29 Quarterly Meeting IU GIG Quarterly Gateway Highlights Q4FY07 Accomplishments –SC 07 LEAD Science Gateway Demonstrations: LEAD Adaptive Cyber Infrastructure LEAD Integration with virtual grid execution system in collaboration with VGrADS LEAD Urgent Computing execution in collaboration with SPRUCE Gateway –Continued work with Globus Developers, TG RP, TG SI, TG GIG & Data WG in testing and debugging latest version of GRAM and Grid FTP which will be rolled out as part of CTSS V4. –Design & Development of Optimal Resource Selection Service which will help gateways select TG resources based on queue wait and estimated file transfer waits based on UCSB NWS system. –Continued discussion with SID Grid, Genome and other potential gateways. –Release of Science Gateway workflow Suite through OGCE project. Q1FY08 Plans –Continue working with Weather Challenge on using LEAD Portal to enable participation from undergraduate students from 67 universities. –Continue to work with TG and Globus teams in testing reliability and scalability of CTSS V4 Grid Middleware. –Support Gateway Software released through OGCE project.


Download ppt "Science Gateways Objectives aka Nancy’s Brave New Gateway World Quarterly Meeting, December 6-7, 2007."

Similar presentations


Ads by Google