TeraGrid Science Gateways Nancy Wilkins-Diehr Area Director for Science Gateways TeraGrid 09 Education Program, June 22, 2009

You in this audience are the future of computational science You can tell me more about the value of gateways than I can tell you First let me give you some background so we’re on the same page, then let’s begin a discussion

Today I hope to demonstrate What are gateways? –Why are gateways worth the effort? –What do they allow scientists to do that they couldn't do without gateways? –What are some specific examples of this? –Why are these examples important? TeraGrid 09 Education Program, June 22, 2009

Gateways A natural result of the impact of the internet on worldwide communication and information retrieval Implications on the conduct of science are still evolving –1980’s, Early gateways, National Center for Biotechnology Information BLAST server, search results sent by , still a working portal today –1992 Mosaic web browser developed –1995 “International Protein Data Bank Enhanced by Computer Browser” –2004 TeraGrid project director Rick Stevens recognized growth in scientific portal development and proposed the Science Gateway Program –Today, Web 3.0 and programmatic exchange of data between web pages Simultaneous explosion of digital information –Growing analysis needs in many, many scientific areas –Sensors, telescopes, satellites, digital images and video –#1 machine on Top500 today is 1000x more powerful than all combined entries on the first list in 1993 Scientists developing their own tools to take advantage of the internet and help address the digital data problem TeraGrid 09 Education Program, June 22, 2009 Only 17 years since the release of Mosaic!

NSF understands the impact technology can have on science “Virtual environments have the potential to enhance collaboration, education, and experimentation in ways that we are just beginning to explore.” “In every discipline, we need new techniques that can help scientists and engineers uncover fresh knowledge from vast amounts of data generated by sensors, telescopes, satellites, or even the media and the Internet.” TeraGrid 09 Education Program, June 22, 2009 Gateways are a terrific example of interfaces that can support transformative science

Today, there are approximately 35 gateways using the TeraGrid TeraGrid 09 Education Program, June 22, 2009

Why are gateways worth the effort? Increasing range of expertise needed to tackle the most challenging scientific problems –How many details do you want each individual scientist to need to know? PBS, RSL, Condor Coupling multi-scale codes Assembling data from multiple sources Collaboration frameworks TeraGrid 09 Education Program, June 22, 2009 #! /bin/sh #PBS -q dque #PBS -l nodes=1:ppn=2 #PBS -l walltime=00:02:00 #PBS -o pbs.out #PBS -e pbs.err #PBS -V cd /users/wilkinsn/tutorial/exercise_3../bin/mcell nmj_recon.main.mdl +( &(resourceManagerContact="tg- login1.sdsc.teragrid.org/jobmanager-pbs") (executable="/users/birnbaum/tutorial/bin/mcell") (arguments=nmj_recon.main.mdl) (count=128) (hostCount=10) (maxtime=2) (directory="/users/birnbaum/tutorial/exercise_3") (stdout="/users/birnbaum/tutorial/exercise_3/globus.out") (stderr="/users/birnbaum/tutorial/exercise_3/globus.err") ) ======= # Full path to executable executable=/users/wilkinsn/tutorial/bin/mcell # Working directory, where Condor-G will write # its output and error files on the local machine. initialdir=/users/wilkinsn/tutorial/exercise_3 # To set the working directory of the remote job, we # specify it in this globus RSL, which will be appended # to the RSL that Condor-G generates globusrsl=(directory='/users/wilkinsn/tutorial/exercise_3') # Arguments to pass to executable. arguments=nmj_recon.main.mdl # Condor-G can stage the executable transfer_executable=false # Specify the globus resource to execute the job globusscheduler=tg-login1.sdsc.teragrid.org/jobmanager- pbs # Condor has multiple universes, but Condor-G always uses globus universe=globus # Files to receive sdout and stderr. output=condor.out error=condor.err # Specify the number of copies of the job to submit to the condor queue. queue 1

Not just ease of use What can scientists do that they couldn’t do previously? Linked Environments for Atmospheric Discovery (LEAD) - access to radar data National Virtual Observatory (NVO) – access to sky surveys Ocean Observing Initiative (OOI) – access to sensor data PolarGrid – access to polar ice sheet data SIDGrid – expensive datasets, analysis tools GridChem –coupling multiscale codes How would this have been done before gateways? TeraGrid 09 Education Program, June 22, 2009

Gateways democratize access to high end resources Almost anyone can investigate scientific questions using high end resources –Not just those in the research groups who log in to supercomputers –Gateways allow anyone with a web browser to explore Opportunities can be uncovered via google –My 11-year-old son discovered nanoHUB.org himself while his class was studying Bucky Balls Foster new ideas, cross-disciplinary approaches –Encourage students to experiment But used in production too –Significant number of papers resulting from gateways including GridChem, nanoHUB –Scientists can focus on challenging science problems rather than challenging infrastructure problems TeraGrid 09 Education Program, June 22, 2009

Tremendous Opportunities Using the Largest Shared Resources - Challenges too! What’s different when the resource doesn’t belong just to me? –Resource discovery –Accounting –Security –Proposal-based requests for resources (peer-reviewed access) Code scaling and performance numbers Justification of resources Gateway citations Tremendous benefits at the high end, but even more work for the developers Potential impact on science is huge –Small number of developers can impact thousands of scientists –But need a way to train and fund those developers and provide them with appropriate tools TeraGrid 09 Education Program, June 22, 2009

Gateways in the marketplace Kids control telescopes and share images “In seconds my computer screen was transformed into a live telescopic view” –“Slooh's users include newbies and professional astronomers in 70 countries” Observatories in the Canary Islands and Chile, Australia coming soon 5000 images/month since 2003 Increases public support for investment in these facilities TeraGrid 09 Education Program, June 22, 2009

Learn about the jet stream using LEAD Can be done with live weather data portal.leadproject.org Education link –LEAD to learn modules Polar jet stream TeraGrid 09 Education Program, June 22, 2009

North American Mesoscale (NAM) model –Two jet streams in North America – polar and subtropical, polar is more prominent –Polar generally moves west to east, but sometimes forms large loops to the north and south First image is a view of wind speed at 45, 55, 65, and 75 m/s. Also can see polar jet stream’s effect on surface temperatures TeraGrid 09 Education Program, June 22, 2009

Move ahead to 6z (1 AM EST) on Jan 30 th Uncheck the box next to the 55 m/s wind speed and the 45 m/s wind speed Looking at the 300 mb pressure and surface temperature, notice the upper-level ridge in the western part of the country and the trough in the eastern part of the country Western states are experiencing warmer temperatures than the eastern states Polar jet stream appears to be in the area of the largest surface temperature gradients and where the 300 mb geopotential height contours are tightest, making it more important to forecasters than the sub-tropical jet stream During the winter months, the surface temperature gradients are strongest and farthest south, causing the polar jet stream to be stronger in the winter than the summer. TeraGrid 09 Education Program, June 22, 2009

Move ahead to 18z (1 PM EST) on January 30 th Uncheck the box next to the 65 m/s wind speed and the Wind Barbs, check the box next to Upward Motion Regions of upward motion at the entrance and exit of the high speed “jet streak” support the development of storm systems Test your knowledge – u/~lead/JS_Questions.html TeraGrid 09 Education Program, June 22, 2009

Social Informatics Data Grid Collaborative access to large, complex datasets SIDGrid is unique among social science data archive projects –Streaming data which change over time Voice, video, images (e.g. fMRI), text, numerical (e.g. heart rate, eye movement) –Login and click portal –Search for "elan example" Public project with video,audio and annotation data Preview the data with Flash Search Export the data –Can visualize and annotate with SIDGrid Elan software » out/getting_startedhttps://sidgrid.ci.uchicago.edu/ab out/getting_started Run the analysis scripts. TeraGrid 09 Education Program, June 22,

Gateways and Gadgets Web technologies change fast –Must be able to adapt quickly Gateways and gadgets –Gateway components incorporated into any social networking page –75% of 18 to 24 year-olds have social networking websites iPhone apps? Web 3.0 –Beyond social networking and sharing content –Standards and querying interfaces to programmatically share data across sites Resource Description Framework (RDF), SPARQL TeraGrid 09 Education Program, June 22, 2009

Open Life Sciences Gateway Add a BLAST query to your igoogle page and run on the TeraGrid Google.com, add stuff Gadget authorization –Click the link of “Personalize the gadget” –Use account/password scidac/scidac to login in Here is a short test sequence: >MRLFKTRKSTDTYSTLAAQQQQQQQQQQQHQAEGSNISHSSNSSSNKSHT PATCSNRLNKSIVSSTSISSSLPDLHDKSPVMILSCTTLASNGATATAAV TATATGTAATSGGSLQQQQQQHLQHQQQQQPLRTATPTCLLSGRQTPSAI Check the blast type to "blastp" and leave the job submission site as "any site". Click OK, you will find a new blast job is created on Job History Portlet (Gadget). TeraGrid 09 Education Program, June 22, 2009

Tremendous Potential for Gateways In only 17 years, the Web has fundamentally changed human communication Science Gateways can leverage this amazingly powerful tool to: –Transform the way scientists collaborate –Streamline conduct of science –Influence the public’s perception of science Reliability, trust, continuity are fundamental to truly change the conduct of science through the use of gateways –High end resources can have a profound impact We need you to help shape this very exciting future! TeraGrid 09 Education Program, June 22, 2009

Now that you know more about gateways, what ideas come to mind for gateways in your area of interest?

TeraGrid 09 Education Program, June 22, 2009 Have fun at TG09 Contact me any time Nancy Wilkins-Diehr,