University of Illinois at Chicago Lambda Grids and The OptIPuter Tom DeFanti

University of Illinois at Chicago What is The OptIPuter? The OptIPuter is a NSF Large ITR Award –UCSD Cal-(IT) 2 ’s Larry Smarr, PI, Phil Papadopoulos, Mark Ellisman, John Orcutt, Andrew Chien, Greg Hidley –UIC EVL’s Tom DeFanti, Jason Leigh, and Maxine Brown –And USC, UCI, SDSU, NU, TAMU, UvA partners NIH Biomedical Informatics Research Network NSF EarthScope siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml

University of Illinois at Chicago The OptIPuter is Driven by Data-Intensive Scientific Applications that will Need Optical Networks Large Data Challenges in Neuro and Earth Sciences –Each Data Object is 3D and Gigabytes –Data are Generated and Stored in Distributed Archives –Research is Carried Out on Federated Repository Requirements –Computing Requirements  PC Clusters –Communications  Dedicated Lambdas Over Fiber –Data  Large Peer-to-Peer Lambda Attached Storage –Visualization  Collaborative Volume Algorithms Response –OptIPuter Research Plan

University of Illinois at Chicago Mod 1 OptIPuter (a.k.a. Geowall2): Extremely High Resolution Scalable Digital Montage Visualization USGS EROS Center Data: –133 urban areas: –5643 tiles –Each 5000x5000 pixels –~ 375,600 x 375,600 pixels each urban area –394GigaBytes per area –Total data ~51 TeraBytes Hi-Tech Coast GIS Project –Link GIS of San Diego and Orange County with Baja –Anchor with High Resolution Montage of Aerial Photos Source: Jason Leigh, EVL, UIC

University of Illinois at Chicago Bandwidth is getting cheaper faster than storage. Storage is getting cheaper faster than computing. The exponentials growth curves are crossing… “A global economy designed to waste transistors, power, and silicon area -and conserve bandwidth above all- is breaking apart and reorganizing itself to waste bandwidth and conserve power, silicon area, and transistors." George Gilder Telecosm (2000) The Gilder/OptIPuter Philosophy

University of Illinois at Chicago Optical Networks Are Emerging as the 21 st Century Driver Scientific American, January 2001 Number of Years Performance per Dollar Spent Data Storage (bits per square inch) (Doubling time 12 Months) Optical Fiber (bits per second) (Doubling time 9 Months) Silicon Computer Chips (Number of Transistors) (Doubling time 18 Months)

University of Illinois at Chicago The OptIPuter Hardware: 2003 Experimental Network, Wide Array of Vendors Glimmerglass

University of Illinois at Chicago Non-Trivial Problems to Solve The Internet is not designed for single large- scale users—classic TCP is not usable for long fat applications Circuits are not scalable, but neither are router$ All intelligence has to be on the edge Tuning compute, data, visualization, networking using clusters to get a simple order of magnitude improvement Security at 10Gb line speed

University of Illinois at Chicago How Can We Afford Lambdas? 30 years ago, acoustic 300b USA to Netherlands cost US$4.00/minute Now, OC-192 (10Gb) costs US$2.00/minute* (a GigE costs the same as a cell phone call) That’s 2,500,000 times cheaper *if you buy it for 525,600 minutes and manage it yourself

University of Illinois at Chicago Which Users can Justify 25¢/minute Lambdas? DSLGigE LAN C A B A -> Need full Internet routing B -> Need VPN services on/and full Internet routing C -> Need very fat pipes, limited multiple Virtual Organizations Source: Cees de Laat Number of users Bandwidth consumed

University of Illinois at Chicago What are Lambda Grids? High-performance trials of new technologies that support application-dictated development of software toolkits, middleware, computing and networking. Provide known and knowable characteristics with deterministic and repeatable behavior on a persistent basis, while encouraging experimentation with innovative concepts. The missing link between Research and Production Networks? Metro, Regional, National, International Scale

University of Illinois at Chicago UIC Chicago Metro Scale Lambdas Metro GigE’s $30K one-time cost

University of Illinois at Chicago Illinois’ I-WIRE Regional-Scale Lambdas I-WIRE: $7.5M for 20-year leases Similar efforts in many states ANL NCSA/UIUC UIC ANL UIUC UC NU/BN StarLight IIT