The OptIPuter – From SuperComputers to SuperNetworks Vanguard NextGens Conference Coronado, CA November 19, 2002 Dr. Larry Smarr Director, California Institute.

Slides:

Advertisements

Similar presentations

Electronic Visualization Laboratory University of Illinois at Chicago EVL Optical Networking Research Oliver Yu Electronic Visualization Laboratory University.

Advertisements

Grids and Biology: A Natural and Happy Pairing Rick Stevens Director, Mathematics and Computer Science Division Argonne National Laboratory Professor,

-Grids and the OptIPuter Software Architecture Andrew A. Chien Director, Center for Networked Systems SAIC Chair Professor, Computer Science and Engineering.

University of Illinois at Chicago The Future of STAR TAP: Enabling e-Science Research Thomas A. DeFanti Principal Investigator, STAR TAP Director, Electronic.

OptIPuter Goal: Removing Bandwidth Barriers to e-Science ATLAS Sloan Digital Sky Survey LHC ALMA.

University of Illinois at Chicago Annual Update Thomas A. DeFanti Principal Investigator, STAR TAP Director, Electronic Visualization Laboratory.

California Networking Going To OPTICAL NETWORKING John Silvester Chair, CENIC and University of Southern California Pacific Rim Networking 2/21/02.

Creating High Performance Lambda Collaboratories" ONR Briefing ACCESS DC Arlington, VA March 25, 2005 Dr. Larry Smarr Director, California Institute for.

Calit2-Living in the Future " Keynote Sharecase 2006 University of California, San Diego March 29, 2006 Dr. Larry Smarr Director, California Institute.

Bringing Mexico Into the Global LambdaGrid Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber.

Technology Trends Shaping the Classroom of the Future

The Coming Transformation of the Internet Invited Talk to Mission Ventures Annual Meeting for Investors The Ritz-Carlton Laguna Niguel, CA March 20, 2001.

Why Optical Networks Are Emerging as the 21 st Century Driver Scientific American, January 2001.

The California Institute for Telecommunications and Information Technology Briefing to the Bandwidth Bay Committee Downtown San Diego, CA April 4, 2001.

Cal-(IT) 2 and Homeland Security Jacobs School of Engineering Council of Advisors San Diego Yacht Club San Diego, CA November 22, 2002 Dr. Larry Smarr.

Cal-(IT) 2 – The Impact on San Diego and the High Tech Coast Invited Talk for the University of The Third Age University of San Diego San Diego, CA January.

Applications Requiring An Experimental Optical Network Invited Keynote I-Light Applications Workshop Indiana Univ. Purdue Univ. Indianapolis December 4,

The Drive Toward Dedicated IP Lightpipes for e-Science Applications OSAs 6th Annual Photonics & Telecommunications Executive Forum Panel on "Back to the.

California Institute for Telecommunications and Information Technology Invited Talk in the Research Review 2001 Jacobs School of Engineering, UCSD San.

Briefing on Cal-(IT) 2 Presentation to the California Office of Emergency Services CIO Team UCSD Scripps Institute of Oceanography La Jolla, CA February.

Back to the FutureThe Increasing Importance of the States in Setting the Research Agenda Lecture in the Series Defining Values of Research and Technology:

High Performance Collaboration Grid Systems Symposium on Knowledge Environments for Science and Engineering: Past, Present, and Future National Science.

"The OptIPuter: an IP Over Lambda Testbed" Invited Talk NREN Workshop VII: Optical Network Testbeds (ONT) NASA Ames Research Center Mountain View, CA August.

The OptIPuter Lambda Coupled Distributed Computing, Peer-to-Peer Storage, and Volume Visualization Dr. Larry Smarr, Director, California Institute for.

The First Year of Cal-(IT) 2 Report to The University of California Regents UCSF San Francisco, CA March 13, 2002 Dr. Larry Smarr Director, California.

Scaling-Up the BIRN Building on the BIRN Workshop National Institutes of Health Bethesda, MD March 22, 2004 Dr. Larry Smarr Director, California Institute.

Building Living Laboratories of the Future Invited Plenary Talk The Society for College and University Planning San Diego, CA July 17, 2002 Dr. Larry Smarr.

The New Internet and Biomedical Research Briefing to the Deans Advisory Board School of Medicine, UCSD San Diego, CA March 1,

Metro-Area High Performance Networks and Emergency Preparedness Invited Talk to SBC Technology Resources, Inc. Austin, TX December 4, 2001 Larry Smarr,

OptIPuter and ENDfusion- Eliminating Bandwidth as an Obstacle in Data Intensive Sciences 21 st NORDUnet Networking Conference Reykjavik, Iceland August.

Technology Trends: New Capabilities in the Next Five Years CalTRANS Visit UCSD September 26, 2002 Dr. Larry Smarr Director, California Institute for Telecommunications.

21st Century Networking and Applications

StarLight Located in Abbott Hall, Northwestern University’s Chicago Campus Operational since summer 2001, StarLight is a 1GigE and 10GigE switch/router.

AHM Overview OptIPuter Overview Third All Hands Meeting OptIPuter Project San Diego Supercomputer Center University of California, San Diego January 26,

Report on Cal-(IT) 2 UCSD Foundation Staff UCSD November 12, 2002 Dr. Larry Smarr Director, California Institute for Telecommunications and Information.

Cal-(IT) 2 : A Public-Private Partnership in Southern California Tech Coast Angels Invited Talk November 11, 2003 Faculty Club, UC San Diego Dr. Larry.

How Will the Future Internet Impact Real Estate? Keynote Address National Association of Industrial and Office Properties San Diego, CA May 16, 2002 Dr.

IGrid Workshop: September 26-29, 2005 GLIF Meeting: September 29-30, 2005 Maxine Brown and Tom DeFanti, Co-Chairs Larry Smarr and Ramesh Rao, Hosts Calit2.

TeraGrid and I-WIRE: Models for the Future? Rick Stevens and Charlie Catlett Argonne National Laboratory The University of Chicago.

Why Optical Networks Will Become the 21 st Century Driver Scientific American, January 2001 Number of Years Performance per Dollar Spent Data Storage.

Developing a North American Global LambdaGrid Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E.

Cal-(IT) 2 : A Public-Private Partnership in Southern California U.S. Business Council for Sustainable Development Year-End Meeting December 11, 2003 Institute.

Chiaro’s Enstara™ Summary Scalable Capacity –6 Tb/S Initial Capacity –GigE  OC-192 Interfaces –“Soft” Forwarding Plane With Network Processors For Maximum.

Distributed Cyberinfrastructure Supporting the Chemical Sciences and Engineering Challenges for the Chemical Sciences in the 21 st Century Workshop on.

Innovative Research Alliances Invited Talk IUCRP Fellows Seminar UCSD La Jolla, CA July 10, 2006 Dr. Larry Smarr Director, California Institute for Telecommunications.

A Wide Range of Scientific Disciplines Will Require a Common Infrastructure Example--Two e-Science Grand Challenges –NSF’s EarthScope—US Array –NIH’s Biomedical.

Using Photonics to Prototype the Research Campus Infrastructure of the Future: The UCSD Quartzite Project Philip Papadopoulos Larry Smarr Joseph Ford Shaya.

University Digital Art Driven Innovation with Industry New Models for Art & Industry Collaboration UCLA Arrowhead Conference Center Lake Arrowhead, CA.

SoCal Infrastructure OptIPuter Southern California Network Infrastructure Philip Papadopoulos OptIPuter Co-PI University of California, San Diego Program.

Cal-(IT) 2 : A Public-Private Partnership in Southern California UCSD CONNECT Leadership Council Invited Talk September 17, 2003 ResMed Facilities, Poway,

The Interaction of UCSD Industrial Partners, the Jacobs School of Engineering, and Cal-(IT) 2 Dr. Larry Smarr Director, California Institute for Telecommunications.

The California Institute for Telecommunications and Information Technology Invited Speaker Council on Competitiveness 2 nd National Innovation Summit:

The OptIPuter – From SuperComputers to SuperNetworks GEON Meeting San Diego Supercomputer Center, UCSD La Jolla, CA November 19, 2002 Dr. Larry Smarr Director,

Ocean Sciences Cyberinfrastructure Futures Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technologies Harry E.

The Coming Transformation of the Internet Invited Keynote to the UCSD CONNECT Financial Forum Sheraton Hotel at Harbor Island San Diego, CA February 21,

The California Institute for Telecommunications and Information Technology Larry Smarr Institute Director UCSD October 12, 2001.

University of Illinois at Chicago StarLight: Applications-Oriented Optical Wavelength Switching for the Global Grid at STAR TAP Tom DeFanti, Maxine Brown.

The Future from the Perspective of The California Institute for Telecommunications and Information Technology Invited Paper to the Eighteenth IEEE Symposium.

The OptIPuter Project Tom DeFanti, Jason Leigh, Maxine Brown, Tom Moher, Oliver Yu, Bob Grossman, Luc Renambot Electronic Visualization Laboratory, Department.

The California Institute for Telecommunications and Information Technology Briefing to the First Summit of the California Institutes for Science and Innovation,

OptIPuter Networks Overview of Initial Stages to Include OptIPuter Nodes OptIPuter Networks OptIPuter Expansion OPtIPuter All Hands Meeting February 6-7.

“ Ultra-Broadband and Peta-Scale Collaboration Opportunities Between UC and Canada Summary Talk Canada - California Strategic Innovation Partnership Summit.

“The UCSD Big Data Freeway System” Invited Short Talk Workshop on “Enriching Human Life and Society” UC San Diego February 6, 2014 Dr. Larry Smarr Director,

Southern California Infrastructure Philip Papadopoulos Greg Hidley.

University of Illinois at Chicago Lambda Grids and The OptIPuter Tom DeFanti.

NSF International Research Network Connections (IRNC) Program: TransLight/StarLight Maxine D. Brown and Thomas A. DeFanti Electronic Visualization Laboratory.

“OptIPuter: From the End User Lab to Global Digital Assets" Panel UC Research Cyberinfrastructure Meeting October 10, 2005 Dr. Larry Smarr.

Charlie Catlett UIUC/NCSA Starlight International Optical Network Hub (NU-Chicago) Argonne National Laboratory U Chicago IIT UIC.

Maxine Brown, Tom DeFanti, Joe Mambretti

Optical SIG, SD Telecom Council

Presentation transcript:

The OptIPuter – From SuperComputers to SuperNetworks Vanguard NextGens Conference Coronado, CA November 19, 2002 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technologies Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD

California Has Initiated Four New Institutes for Science and Innovation UCSB UCLA California NanoSystems Institute UCSF UCB California Institute for Bioengineering, Biotechnology, and Quantitative Biomedical Research UCI UCSD California Institute for Telecommunications and Information Technology Center for Information Technology Research in the Interest of Society UCSC UCD UCM

Cal-(IT) 2 An Integrated Approach to the Future Internet UC San Diego & UC Irvine Faculty Working in Multidisciplinary Teams With Students, Industry, and the Community The States $100 M Creates Unique Buildings, Equipment, and Laboratories

Over Fifty Industrial Sponsors From a Broad Range of Industries Akamai Technologies Inc. AMCC Ampersand Ventures Arch Ventures Avalon Ventures The Boeing Company Broadcom Corporation CAIMIS, Inc. Chiaro Networks Conexant Systems, Inc. Connexion by Boeing Cox Communications Diamondhead Ventures Dupont Emulex Corporation Network Systems Enosys Markets Enterprise Partners Entropia, Inc. Ericsson ESRI Extreme Networks Global Photon Systems Graviton IBM Computers Communications Software Sensors Biomedical Startups Venture Capital Newport Corporation Oracle Orincon Industries Panoram Technologies Printronix QUALCOMM Quantum The R.W. Johnson Pharmaceutical Research Institute SAIC Samueli, Henry (Broadcom) SciFrame, Inc. Seagate Storage Products SGI Silicon Wave Sony STMicroelectronics, Inc. Sun Microsystems TeraBurst Networks Texas Instruments Time Domain UCSD Healthcare WebEx IdeaEdge Ventures The Irvine Company Intersil Corporation Irvine Sensors Corporation JMI, Inc. Leap Wireless International Link, William J. (Versant Ventures) Litton Industries, Inc. MedExpert International Merck Microsoft Corporation Mission Ventures NCR $140 Million in Industrial Matching

Closing in on the Dream Using satellite technology…to demo what It might be like to have high-speed fiber-optic links between advanced computers in two different geographic locations. Al Gore, Senator Chair, US Senate Subcommittee on Science, Technology and Space What we really have to do is eliminate distance between individuals who want to interact with other people and with other computers. Larry Smarr, Director National Center for Supercomputing Applications, UIUC SIGGRAPH 89 Science by Satellite Source: Maxine Brown, EVL, UIC Boston Illinois

The Move to Data-Intensive Science & Engineering- e-Science Community Resources ATLAS Sloan Digital Sky Survey LHC ALMA

Four Large Hadron Collider Experiments: The Petabyte to Exabyte Challenge ATLAS, CMS, ALICE, LHCB Higgs + New particles; Quark-Gluon Plasma; CP Violation Data stored ~40,000 Terabytes/Year and UP Data stored ~40,000 Terabytes/Year and UP Source: Harvey Newman, Caltech

NIH is Funding a Brain Imaging Federated Repository National Partnership for Advanced Computational Infrastructure Part of the UCSD CRBS Center for Research on Biological Structure Biomedical Informatics Research Network (BIRN) NIH Plans to Expand to Other Organs and Many Laboratories

NSFs EarthScope Rollout Over 14 Years Starting With Existing Broadband Stations

The Grid is the Emerging Distributed Cyberinfrastructure Middleware Applications and Capabilities Toolkits/Workbenches and Portals Distributed Computing Collaboration Visualizatio n Data Mining Instrument s / Sensors Problem Solving Environment s GRID Services (Middleware) Authenticatio n / Authorization Resource Discovery Schedulin g / Allocation Fault Detection / Recovery Event Services Information Infrastructur e GRID Fabric (Resources) QoS / Diffserv Directory Services Informatio n Mgmt / Storage Public Key Infrastructure Site accounting / scheduling Operating Systems Open Grid Services Architecture (OGSA) Integrates Globus with Web Services

Grid Computing is Becoming Mainstream

Why Optical Networks Are Emerging as the 21 st Century Driver for the Grid Scientific American, January 2001

The Crossing Exponentials Requires a Rethinking of Grid Architecture A global economy designed to waste transistors, power, and silicon area -and conserve bandwidth above all- is breaking apart and reorganizing itself to waste bandwidth and conserve power, silicon area, and transistors." George Gilder Telecosm (2000)

The Rapid Increase in Bandwidth is Driven by Parallel Lambdas On Single Optical Fibers (WDM) Parallel Lambdas Will Drive This Decade The Way Parallel Processors Drove the 1990s

CONTROLPLANECONTROLPLANE Clusters Dynamically Allocated Lightpaths Switch Fabrics Physical Monitoring Apps Middleware A LambdaGrid Will Be the Backbone for an e-Science Network Source: Joe Mambretti, NU

The Next S-Curves of Networking Exponential Technology Growth 0% 100% Research Experimental/ Early Adopters Production/ Mass Market Time Technology S-Curve Gigabit Testbeds Connections Program Internet2 Abilene DWDM Experimental Networks Lambda Grids ~1990s Networking Technology S-Curves Technology Penetration

Data Intensive Scientific Applications Require Experimental Optical Networks Large Data Challenges in Neuro and Earth Sciences –Each Data Object is 3D and Gigabytes –Data are Generated and Stored in Distributed Archives –Research is Carried Out on Federated Repository Requirements –Computing Requirements PC Clusters –Communications Dedicated Lambdas Over Fiber –Data Large Peer-to-Peer Lambda Attached Storage –Visualization Collaborative Volume Algorithms Response –OptIPuter Research Project

UIUC/NCSA Starlight (NU-Chicago) Argonne UChicago IIT UIC Illinois Century Network James R. Thompson Ctr City Hall State of IL Bldg 4 pair 12 pair 4 pair 2 pair 4 pair 18 pair 410 pair 12 pair 2 pair Level(3) 111 N. Canal McLeodUSA 151/155 N. Michigan Doral Plaza Qwest 455 N. Cityfront UC Gleacher 450 N. Cityfront Illinois I-WIRE The First State Dark Fiber Experimental Network Source: Charlie Catlett 12/2001

Fifteen Countries/Locations Proposing 28 Demonstrations: Canada, CERN, France, Germany, Greece, Italy, Japan, The Netherlands, Singapore, Spain, Sweden, Taiwan, United Kingdom, United States Applications Demonstrated: Art, Bioinformatics, Chemistry, Cosmology, Cultural Heritage, Education, High-Definition Media Streaming, Manufacturing, Medicine, Neuroscience, Physics, Tele-science Grid Technologies: Grid Middleware, Data Management/ Replication Grids, Visualization Grids, Computational Grids, Access Grids, Grid Portal iGrid 2002 September 24-26, 2002, Amsterdam, The Netherlands UIC Sponsors: HP, IBM, Cisco, Philips, Level (3), Glimmerglass, etc.

iGrid 2002 Was Sustaining 1-3 Gigabits/s Total Available Bandwidth Between Chicago and Amsterdam Was 30 Gigabit/s

The NSF TeraGrid A LambdaGrid of Linux SuperClusters NCSA 8 TF 4 TB Memory 240 TB disk Caltech 0.5 TF 0.4 TB Memory 86 TB disk Argonne 1 TF 0.25 TB Memory 25 TB disk TeraGrid Backbone (40 Gbps) SDSC 4.1 TF 2 TB Memory 250 TB disk This will Become the National Backbone to Support Multiple Large Scale Science and Engineering Projects Data Compute Visualization Applications Intel, IBM, Qwest Myricom, Sun, Oracle $53Million from NSF

From SuperComputers to SuperNetworks-- Changing the Grid Design Point The TeraGrid is Optimized for Computing –1024 IA-64 Nodes Linux Cluster –Assume 1 GigE per Node = 1 Terabit/s I/O –Grid Optical Connection 4x10Gig Lambdas = 40 Gigabit/s –Optical Connections are Only 4% Bisection Bandwidth The OptIPuter is Optimized for Bandwidth –32 IA-64 Node Linux Cluster –Assume 1 GigE per Processor = 32 gigabit/s I/O –Grid Optical Connection 4x10GigE = 40 Gigabit/s –Optical Connections are Over 100% Bisection Bandwidth

The OptIPuter is an Experimental Network Research Project Driven by Large Neuroscience and Earth Science Data Multiple Lambdas Linking Clusters and Storage –LambdaGrid Software Stack –Integration with PC Clusters –Interactive Collaborative Volume Visualization –Lambda Peer to Peer Storage With Optimized Storewidth –Enhance Security Mechanisms –Rethink TCP/IP Protocols NSF Large Information Technology Research Proposal –UCSD and UIC Lead CampusesLarry Smarr PI –USC, UCI, SDSU, NW Partnering Campuses –Industrial Partners: IBM, Telcordia/SAIC, Chiaro Networks –$13.5 Million Over Five Years

The OptIPuter is an Experimental Network Research Project Driven by Large Neuroscience and Earth Science Data Multiple Lambdas Linking Clusters and Storage –LambdaGrid Software Stack –Integration with PC Clusters –Interactive Collaborative Volume Visualization –Lambda Peer to Peer Storage With Optimized Storewidth –Enhance Security Mechanisms –Rethink TCP/IP Protocols NSF Large Information Technology Research Proposal –UCSD and UIC Lead CampusesLarry Smarr PI –USC, UCI, SDSU, NW Partnering Campuses –Industrial Partners: IBM, Telcordia/SAIC, Chiaro Networks –$13.5 Million Over Five Years

Metro Optically Linked Visualization Walls with Industrial Partners Set Stage for Federal Grant Driven by SensorNets Data –Real Time Seismic –Environmental Monitoring –Distributed Collaboration –Emergency Response Linked UCSD and SDSU –Dedication March 4, 2002 Linking Control Rooms Cox, Panoram, SAIC, SGI, IBM, TeraBurst Networks SD Telecom Council UCSD SDSU 44 Miles of Cox Fiber

OptIPuter NSF Proposal Partnered with National Experts and Infrastructure Vancouver Seattle Portland San Francisco Los Angeles San Diego (SDSC) NCSA SURFnet CERN CA* net4 Asia Pacific Asia Pacific AMPATH PSC Atlanta CA*net4 Source: Tom DeFanti and Maxine Brown, UIC NYC TeraGrid DTFnet CENIC Pacific Light Rail Chicago UIC NU USC UCSD, SDSU UCI

switch Cluster – Disk Disk – Disk Viz – Disk DB – Cluster Cluster – Cluster Medical Imaging and Microscopy Chemistry, Engineering, Arts San Diego Supercomputer Center Scripps Institution of Oceanography Chiaro Enstara OptIPuter LambdaGrid Enabled by Chiaro Networking Router

½ Mile The UCSD OptIPuter Deployment SIO SDSC CRCA Phys. Sci - Keck SOM JSOE Preuss 6 th College Phase I, Fall 02 Phase II, 2003 SDSC Annex To Other OptIPuter Sites Collocation point Node M The UCSD OptIPuter LambdaGrid Testbed Earth Sciences SDSC Arts Chemistry Medicine Engineering High School Undergrad College Phase I, Fall 02 Phase II, 2003 SDSC Annex To Other OptIPuter Sites Collocation point Collocation

Fast polygon and volume rendering with stereographics GeoWall Earth Science GeoFusion GeoMatrix Toolkit Underground Earth Science Rob Mellors and Eric Frost, SDSU SDSC Volume Explorer Dave Nadeau, SDSC, BIRN SDSC Volume Explorer Neuroscience Anatomy Visible Human Project NLM, Brooks AFB, SDSC Volume Explorer 3D APPLICATIONS: + = OptIPuter Transforms Individual Laboratory Visualization, Computation, & Analysis Facilities The Preuss School UCSD OptIPuter Facility