1. Victoria A. White Head, Computing Division Fermilab white@fnal.gov Global Scientific Collaborations and Grids: A View from Fermilab

2. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White2 The 3 Top issues for Particle Physics today (1) Collaboration (2) Collaboration (3) Collaboration › Global within the field - needed for every stage of an experiment  to build the detector and the software, operate and monitor the experiment, produce Monte Carlo data, process data, analyze data, participate in the physics › With other sciences – for Funding need recognition and partnerships outside our field. Also to train the next generation of scientists. › With Educators and Communicators – For public education and to make a difference in society  This is needed worldwide – on both sides of the digital divide

3. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White3 The Fermilab Experimental Program Statistics “Active” experiments from the 2003 Research Program Workbook › Of 213 Institutions involved 114 of them are non-US › Of 1916 physicists, 753 are non-US › Of 699 students, 234 are non-US

4. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White4 Significant Contributions to the Fermilab Program from the “other side” of the digital divide › Brasil › Argentina › Colombia › Mexico › Czech Republic › Taiwan › Russia › China › Korea › India › …. and others

5. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White5 Highlighting Brasil (1) History of truly significant contributions to Fermilab Experiments since 1983  Series of charm and heavy flavor expts E791 was ahead of its time in volume of data Analyzed about 10% of E791 data in Rio - one of only 4 institutions - Ohio, Mississipi, Fermilab, CBPF Significant contribution to the physics - data led to the 1st publication for E791. Focus  Selex CBPF/LAFEX + Fermilab hosted CHEP95 – pushing networking and videoconferencing technology

6. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White6 Highlighting Brasil (2) › Most recently big contributions to the D0 experiment  Designed and built the POTS for the Forward Proton Detector giving the D0 expt a new physics reach  Physics Leadership – far beyond the size of the effort › Leon Lederman, Fermilab Director Emeritus, has throughout provided support and encouragement for Physics in South America

7. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White7 IT Technology and Physics › Network connectivity and IT infrastructure must be available at some minimal level in order to participate in a Physics Experiment.  Being on the wrong side of the digital divide is painful › We need to help ensure that governments understand  Just how important this contribution is to the Science &  THAT HIGH ENERGY PHYSICS IS IN THE FOREFRONT OF SHOWING HOW GLOBAL E-SCIENCE CAN BE CARRIED OUT AND OF DEMONSTRATING THE USE OF TECHNOLOGIES AND BEHAVIORS THAT WILL PROFOUNDLY AFFECT EDUCATION AND COMMERCE AND SOCIETY.

8. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White8 Fermilab Ongoing Scientific Program, 2004 › Run II (CDF, D0, Accelerator. Upgrades) *** › MINOS * › MiniBoone › Sloan Digital Sky Survey ** › CMS ***** › Lattice QCD Theory collaboration ** › Pierre Auger and CDMS › Test Beams › BTeV in pre-project stage, preparing for review **** › SNAP, Off-axis neutrinos and a few other efforts in R&D, LOI stages *represents Large Global Collaboration/Petabyte Datasets/Massive distributed computational power needed And R&D for the Linear Collider – a worldwide effort we assume

9. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White9 Inclusive Worldwide Collaboration is essential for Particle Physics What are we doing at Fermilab to help this ?  Networks and Network research  Grids -- lots on this – important strategically  Guest Scientist program  Education and Outreach program  Experiment sociology and leadership – changes  Videoconferencing  Virtual Control Rooms  Physics Analysis Center for CMS  Public Relations

10. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White10 Networks › Starting 1 year ago Fermilab moved aggressively to make a plan for ~3 GigE connectivity in the (fairly near) future  Dark Fiber to Starlight (contract just about in place finally)  Worked on Office of Science Strategic Networking plan Expect Metropolitan Area Network (MAN) in Chicago area linking ANL, Fermilab and Starlight in a ring  Public Planning processes and reports are important – e.g. TAN report, ICFA SCIC We try to participate whenever we are invited to

11. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White11 ESnet in 2003: OC192 and OC48 Links Coming Into Service; Consider Links to US HENP Labs; Evolution Not Sufficient

12. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White12 ESNet proposal for Chicago area MAN (Metropolitan Area Network)

13. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White13  

14. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White14 Network Research › Also in the past year Fermilab chose to become a beginning player in the Network research arena › As Harvey Newman says often raw bandwidth is not enough – we need to learn how to use it, monitor the network, look for research partners and opportunities  Joining with several partners on proposal for UltraNet (DOE research network) exploitation and with Caltech et. al. on Ultralight program of work/proposal  Some international opportunities for broader collaborations UK, CA, Czech, Netherlands, DataTag, ? › Research agenda pushes provisioning of network

15. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White15 Network Monitoring › Modest effort to help with Les Cottrell’s IEPM and Pinger projects › Modest efforts by Fermilab network group to help track down network bottlenecks and problems

16. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White16 Grids for Science › There is no doubt that computing resources for Particle Physics are going to be distributed globally and shared between “Virtual Organizations”  Happening already with Run II, Babar, RHIC and LHC experiments

17. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White17 2000+ Physicists 159 Institutes 36 Countries LHC: Key Driver for Grids › Complexity:Millions of individual detector channels › Scale:PetaOps (CPU), 100s of Petabytes (Data) › Distribution:Global distribution of people & resources CMS Collaboration

18. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White18 Grid Projects and Coordinating Bodies Fermilab is involved in numerous Grid projects and coordination bodies  PPDG, GriPhyN and iVDGL - the Trillium US Grid projects  LCG - GDB, SC2, (PEB until recently), bodies and working groups on Security, Operations Center, Rollout, etc.  SRM – storage systems standards  Joint Technical Board and Interoperability work  Global Grid Forum – Physics Research Area  Global Grid Forum – Security Research Area  Open Science Grid – meta-level collaboration and coordination  And probably some I forgot…..

19. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White19 Fermilab Grid Strategy › Strategy (from a Fermi-centric view)  Common approaches and technologies across our entire portfolio of experiments and projects › Strategy (from a global HEP view)  Work towards common standards and interoperability › At Fermilab we are working aggressively to put all of our Computational and Storage Fabric on “the Grid” and to contribute constructively to interoperability of Grid middleware

20. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White20 Successes – D0 and CDF › SAM-GRID – a functional Grid system developed for D0 and now in use by CDF for Data Storage with meta-data, Data Delivery and caching, Job Execution (JIM) and Monitoring  SAM “stations” worldwide  Lots of real operational experience and solutions for robustness and scalability  Evolving to use Globus and Condor based “standards” Supports GridFTP + other storage access/file Xfr

21. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White21 SAM-GRID System at D0 Registered Users600 Number of Stations56 Registered Nodes900 Total Disk Cache40 TB Number of Files1.5M Regional Center Analysis site Summary of Resources (DØ)Integrated Files Consumed vs Month (DØ) Integrated GB Consumed vs Month (DØ) 4.0 M Files Consumed 1.2 PB Consumed 3/02 3/03

22. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White22 SAM at a Glance – D0 and CDF › http://d0db.fnal.gov/sam_local/SamAtAGlance/ http://d0db.fnal.gov/sam_local/SamAtAGlance/  Includes Stations in U.S., Czech Republic, India, France, UK, Netherlands, Germany, Canada Monitored Drill down to display jobs running and files being delivered › http://cdfdb.fnal.gov/sam_local/SamAtAGlance/ http://cdfdb.fnal.gov/sam_local/SamAtAGlance/

23. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White23 Successes - US-CMS

24. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White24 Grid2003: An Operational Grid - Huge Success !  28 sites (2100-2800 CPUs)  400-1100 concurrent jobs  10 applications  Running since October 2003 Korea http://www.ivdgl.org/grid2003

25. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White25 Grid2003: A Collaborative Effort › Trillium Grid projects  PPDG + GriPhyN + iVDGL  (US-ATLAS, US-CMS, BTEV, LIGO, SDSS, Computer Science) › US-ATLAS and US-CMS projects  Fermilab, LBL, Argonne  U. New Mexico, U. Texas Arlington › Korean site  Kyungpook National University (CMS) › New sites  University of Buffalo (CCR)

26. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White26 Grid2003: 3 Months Usage

27. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White27 Open Science Grid › Goals (http://www.opensciencegrid.org/)  Support US-LHC research program, other scientific efforts  Federate with LCG and peer with EGEE in Europe  Involve laboratories (DOE) and universities (NSF) › Getting there: Grid2004 (OSG-1), Grid2005 (OSG-2), …  Series of releases  increasing functionality & scale  Persistent Production quality Grid + Grid Laboratory › Jan. 12 meeting in Chicago  >60 participants from labs and universities, Computer Science and Physics and Biology, and Grid Projects

28. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White28 Federating Grids As the US representative to the LHC Computing Grid Grid Deployment Board (LCG GDB) I have repeatedly pushed the case for the LHC Computing Grid to be a “Grid of Grids” – not a distributed Computing center  Resources must be able to belong to >1 Grid E.g. NorduGrid, UK Grid, Open Science Grid, TeraGrid, Region XYZ Grid, University ABC Grid and Fermilab Grid!  Grids must learn to interoperate  Governance, policies, security, resource usage and service level agreements must be considered at many levels including at the level of a Federation of Grids

29. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White29 Inclusive Worldwide Collaboration is essential for Particle Physics What are we doing at Fermilab and in Fermilab experiments to help this ?  Networks and Network research  Grids -- lots on this – important strategically  Guest Scientist program  Education and Outreach program  Experiment sociology and leadership – changes  Videoconferencing  Virtual Control Rooms  Physics Analysis Center for CMS  Public Relations

30. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White30 Support for Guest Scientists and Engineers › Although budgets are tight Fermilab (and especially the Computing Division) wants to encourage and support a guest scientist and guest engineer/software engineer program  Tremendously beneficial to the expts  Tremendously important for ongoing collaboration and communication

31. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White31 Education › Quarknet meets “Grid” › Traditional K-12 Education Program at Fermilab is working with Grid Projects and Computing Division on the educational opportunities afforded by the Grid › Open Science Grid – will have a significant Educational component.

32. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White32 Sociology and Leadership changes › Increasing awareness at Fermilab of the need to include those outside Fermilab  Meeting times to take account of time zones  Videoconferencing  Posting materials and documenting  Leaders not resident at Fermilab

33. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White33 Videoconferencing › We do more and more at Fermilab each year – 5 new fully equipped conference rooms last year  Total of 19 rooms - all Polycom based › Fermilab is instituting better “official” support for VRVS and H323-based conferencing.  New server v. soon  Expect to see growth in desktop video in next 2 years  User Requirements meeting soon – to plan next steps

34. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White34 Virtual Control Room › MINOS has one at Fermilab for a detector at the Sudan Mine in Minnesota  Control Room Logbook › US CMS is going to build one at Fermilab  Now we in the US are the ones who will be far from the apparatus and the action!  We hope our work in the next few years will be broad reaching and benefit all who are “far from the experiment”.

35. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White35 CMS Physics Analysis Center(s) in the U.S. › Serious thought going into this now. What is a PAC? › We can’t all go to CERN, all the time, to do physics  Must empower physicists to do analysis away from CERN  Technology is important – networks, videoconferencing, access to data But there is more  What do we need at Fermilab to have a vibrant center for physics where people come to work together both physically and virtually? › The effort we put into this may have far reaching effects on the “democratization of science”.

36. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White36 Public Relations › Judy Jackson, head of Public Affairs at Fermilab, works closely with many other public affairs groups – at CERN, SLAC and elsewhere › Interactions.org website › Forming a Strategic Grid Communications group – presented a plan for this at a recent Open Science Grid meeting

37. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White37 Role of Science in the Information Society Fermilab/SLAC exhibit from SC2003 on Networking/Grids/Storage/Simulatio ns went to the RSIS exhibition in Geneva recently

38. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White38 Inclusive Worldwide Collaboration is essential for Particle Physics What are we doing at Fermilab to help this ?  Networks and Network research  Grids -- lots on this – important strategically  Guest Scientist program  Education and Outreach program  Experiment sociology and leadership – changes  Videoconferencing  Virtual Control Rooms  Physics Analysis Center for CMS  Public Relations

39. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White39 IT Technology and Physics › Network connectivity and IT infrastructure must be available at some minimal level in order to participate in a Physics Experiment.  Being on the wrong side of the digital divide is painful › We need to help ensure that governments everyone understands  Just how important this contribution is to the Science &  THAT HIGH ENERGY PHYSICS IS IN THE FOREFRONT OF SHOWING HOW GLOBAL E-SCIENCE CAN BE CARRIED OUT AND OF DEMONSTRATING THE USE OF TECHNOLOGIES AND BEHAVIORS THAT WILL PROFOUNDLY AFFECT EDUCATION AND COMMERCE AND SOCIETY.

40. Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004Vicky White40 Conclusions › We must keep working on all fronts to collaborate, educate, make the case for Science and for the Information Technology and Networking improvements that are essential for success – everywhere  in particular to ensure that the Digital Divide does not exclude physicists.