1. “Here comes the Grid” Mark Hayes Technical Director - Cambridge eScience Centre NIEeS Summer School 2003

2. In the beginning… "The collection of people, hardware, and software... will become a node in a geographically distributed computer network…. Through the network... all the large computers can communicate with one another. And through them, all the members of the community can communicate with other people, with programs, with data, or with a selected combination of those resources.” J.C.R.Licklider, “The Computer as a Communication Device” Science and Technology, April 1968 The ARPAnet in 1970

3. International connectivity - 1991

4. International connectivity - 1997

5. International bandwidth From “3D geographic network displays” - Cox et al, ACM Sigmod Record - December 1996

6. What does the Internet look like? http://www.cybergeography.org/

7. The World Wide Web Invented at CERN by Tim Berners-Lee in 1989 as a tool for collaboration and information sharing in the particle physics community.

8. Early distributed computing 1.2 million CPU years so far... Brute force attempt to crack strong encryption Protein folding

9. The Grid - 1998 Editors: Foster & Kesselman 700 pages 22 chapters 40 authors Analogy with the electrical power grid - just plug in.

10. The Grid - 2003 Editors: Berman, Hey, Fox 1000 pages 43 chapters 116 authors Applications, data sharing and virtual communities.

11. It’s not just compute cycles... An exponential growth in data from many areas of science.

12. 4 types of Grid CPU intensive cycle scavenging (SETI@home) Data sharing Application provision Human-human interaction (e.g. Access Grid)

13. SETI@home The world’s most powerful computer delivered 52 Teraflops/second yesterday (Earth Simulator is 35 Tflop/s, sum of top 2-10 is 60Tflop/s) Latest Stats http://setiathome.ssl.berkeley.edu/totals.html 6th July 2003 4.5 E+18 flops/day 52 Teraflops/s 3 E+21 ops 3 zeta ops Floating Point Operations 1,226 years 1.5 M years Total CPU time 1.1 M944 MResults received 1,2264,570,474Users Last 24 HoursTotal

14. The data explosion - some big numbers FTP and GREP are not adequate (Jim Gray) CFD turbulence simulations - 100TB BaBar particle physics experiment - 1TB/day CERN LHC will generate 1GB/s or 10PB/year VLBA radio telescope generates 1GB/s today NCBI/EMBL database is “only 0.5TB” but doubling each year brain imaging - 4TB/brain at full colour, 10  m resolution (4PB/brain at 1  m i.e. cellular resolution) Pixar - 100TB/movie

15. Application provision Google - 10K cpus, 2PB database (2 years ago) free email services - HotMail, Yahoo! 2-10PB storage netsolve - numerical algorithms on demand with Matlab & Mathematica plugins renderfarm.net - graphics rendering on demand

16. The Access Grid “...one of the most compelling glimpses into the future I’ve seen since I first saw NCSA Mosaic.” Larry Smarr Ambient mic (tabletop) Presenter mic Presenter camera Audience camera High end video conferencing and collaboration technology. O(100) nodes world wide.

17. 1 day of cpu time 4 GB ram for a day 1 GB of network bandwidth 1 GB of disk storage 10 M database accesses 10 TB of disk access (sequential) 10 TB of LAN bandwidth (bulk) £1 buys...

18. How do you move a terabyte? 14 minutes6172001,920,0009600OC 1922.2 hours1000Gbps 1 day100100 Mpbs 14 hours97631649,000155OC3 2 days2,01065128,00043T3 2 months2,4698001,2001.5T1 5 months360117700.6Home DSL 6 years3,0861,000400.04 Home phone Time/TB $/TB Sent $/Mbps Rent $/month Speed Mbps Context Source: Terascale SneaketNet, Jim Gray et al

19. Compute cycles are (almost) free... by comparison with network costs. -The cheapest and fastest way to move 1TB of data out from CERN is still by FedEx. Though this considers only bandwidth, low latency networks are even more expensive! (MPI over WAN doesn’t work well.) Some consequences

20. A distributed community of users. Tiny network input & output, huge compute requirement. Database access & storage is also expensive, therefore put the computation near the data. What makes a good Grid application?

21. Questions?