The OptIPuter and Its Applications Invited Talk Cyberinfrastructure for Humanities, Arts, and Social Sciences A Summer Institute SDSC UCSD July 26, 2006 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technologies Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD

From Supercomputer–Centric to Supernetwork-Centric Cyberinfrastructure Megabit/s Gigabit/s Terabit/s Network Data Source: Timothy Lance, President, NYSERNet 32x10Gb Lambdas 1 GFLOP Cray2 60 TFLOP Altix Bandwidth of NYSERNet Research Network Backbones T1 Optical WAN Research Bandwidth Has Grown Much Faster Than Supercomputer Speed! Computing Speed (GFLOPS)

The OptIPuter Project – Creating a SuperWeb for Data Intensive Researchers NSF Large Information Technology Research Proposal –Calit2 (UCSD, UCI) and UIC Lead CampusesLarry Smarr PI –Partners: SDSC, USC, SDSU, NCSA, NW, TA&M, UvA, SARA, NASA Goddard, KISTI, AIST, CRC(Canada), CICESE (Mexico) Industrial Partners –IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent $13.5 Million Over Five YearsNow In the Fourth Year NIH Biomedical Informatics NSF EarthScope and ORION Research Network

What is the OptIPuter? Applications Drivers Interactive Analysis of Large Data Sets OptIPuter Nodes Scalable PC Clusters with Graphics Cards IP over Lambda Connectivity Predictable Backplane Open Source LambdaGrid Middleware Network is Reservable Data Retrieval and Mining Lambda Attached Data Servers High Defn. Vis., Collab. SW High Performance Collaboratory See Nov 2003 Communications of the ACM for Articles on OptIPuter Technologies

Dedicated Optical Channels Makes High Performance Cyberinfrastructure Possible (WDM) Source: Steve Wallach, Chiaro Networks Lambdas Parallel Lambdas are Driving Optical Networking The Way Parallel Processors Drove 1990s Computing

National Lambda Rail (NLR) and TeraGrid Provides Cyberinfrastructure Backbone for U.S. Researchers NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout Links Two Dozen State and Regional Optical Networks DOE, NSF, & NASA Using NLR San Francisco Pittsburgh Cleveland San Diego Los Angeles Portland Seattle Pensacola Baton Rouge Houston San Antonio Las Cruces / El Paso Phoenix New York City Washington, DC Raleigh Jacksonville Dallas Tulsa Atlanta Kansas City Denver Ogden/ Salt Lake City Boise Albuquerque UC-TeraGrid UIC/NW-Starlight Chicago International Collaborators NSFs TeraGrid Has 4 x 10Gb Lambda Backbone

Creating a North American Superhighway for High Performance Collaboration Next Step: Adding Mexico to Canadas CANARIE and the U.S. National Lambda Rail

OptIPuter Scalable Adaptive Graphics Environment (SAGE) Allows Integration of HD Streams OptIPortal– Termination Device for the OptIPuter Global Backplane

20 Dual CPU Nodes, Monitors, ~$50,000 1/4 Teraflop, 5 Terabyte Storage, 45 Mega Pixels--Nice PC! Scalable Adaptive Graphics Environment ( SAGE) Jason Leigh, EVL-UIC Source: Phil Papadopoulos SDSC, Calit2

The Worlds Largest Tiled Display Wall HIPerWall Zeiss Scanning Electron Microscope Center of Excellence in Albert Yee, PI Apple Tiled Display Wall Driven by 25 Dual-Processor G5s 50 Apple 30 Cinema Displays 200 Million Pixels of Viewing Real Estate! HDTV Digital Cameras Digital Cinema Falko Kuester and Steve Jenks, PIs Featured in Apple Computers Hot News

3D Videophones Are Here! The Personal Varrier Autostereo Display Varrier is a Head-Tracked Autostereo Virtual Reality Display –30 LCD Widescreen Display with 2560x1600 Native Resolution –A Photographic Film Barrier Screen Affixed to a Glass Panel –The Barrier Screen Reduces the Horizontal Resolution To 640 Lines Cameras Track Face with Neural Net to Locate Eyes The Display Eliminates the Need to Wear Special Glasses Source: Daniel Sandin, Thomas DeFanti, Jinghua Ge, Javier Girado, Robert Kooima, Tom PeterkaEVL, UIC

How Do You Get From Your Lab to the National LambdaRail? Research is being stalled by information overload, Mr. Bement said, because data from digital instruments are piling up far faster than researchers can study. In particular, he said, campus networks need to be improved. High-speed data lines crossing the nation are the equivalent of six-lane superhighways, he said. But networks at colleges and universities are not so capable. Those massive conduits are reduced to two-lane roads at most college and university campuses, he said. Improving cyberinfrastructure, he said, will transform the capabilities of campus-based scientists. -- Arden Bement, the director of the National Science Foundation

To Build a Campus Dark Fiber Network First, Find Out Where All the Campus Conduit Is!

UCSD Campus-Scale Routed OptIPuter with Nodes for Storage, Computation and Visualization

The New Optical Core of the UCSD Campus-Scale Testbed: Evaluating Packet Routing versus Lambda Switching Goals by 2007: >= 50 endpoints at 10 GigE >= 32 Packet switched >= 32 Switched wavelengths >= 300 Connected endpoints Approximately 0.5 TBit/s Arrive at the Optical Center of Campus Switching will be a Hybrid Combination of: Packet, Lambda, Circuit -- OOO and Packet Switches Already in Place Funded by NSF MRI Grant Lucent Glimmerglass Force10

Created by Garrett Hildebrand Modified by Jessica Yu Calit2 Building UCInet 10 GE HIPerWall Los Angeles SPDS Catalyst 3750 in CSI ONS WDM at UCI campus MPOE (CPL) 1 GE DWDM Network Line Tustin CENIC Calren POP UCSD Optiputer Network 10 GE DWDM Network Line Engineering Gateway Building, Catalyst 3750 in 3 rd floor IDF MDF Catalyst 6500 w/ firewall, 1 st floor closet Wave-2: layer-2 GE. UCSD address space /28 Floor 2 Catalyst 6500 Floor 3 Catalyst 6500 Floor 4 Catalyst 6500 Wave-1: UCSD address space NACS-reserved for testing ESMF Catalyst 3750 in NACS Machine Room (Optiputer) Viz Lab Wave 1 1GE Wave 2 1GE is Up and Working Kim-Jitter Measurements This Week!

Calit2/SDSC Proposal to Create a UC Cyberinfrastructure of OptIPuter On-Ramps to TeraGrid Resources UC San Francisco UC San Diego UC Riverside UC Irvine UC Davis UC Berkeley UC Santa Cruz UC Santa Barbara UC Los Angeles UC Merced OptIPuter + CalREN-XD + TeraGrid = OptiGrid Source: Fran Berman, SDSC, Larry Smarr, Calit2 Creating a Critical Mass of End Users on a Secure LambdaGrid

OptIPuter Software Architecture--a Service-Oriented Architecture Integrating Lambdas Into the Grid GTPXCPUDT LambdaStream CEPRBUDP DVC Configuration Distributed Virtual Computer (DVC) API DVC Runtime Library Globus XIO GRAM GSI Distributed Applications/ Web Services Telescience Vol-a-Tile SAGEJuxtaView Visualization Data Services LambdaRAM DVC Services DVC Core Services DVC Job Scheduling DVC Communication Resource Identify/Acquire Namespace Management Security Management High Speed Communication Storage Services IP Lambdas Discovery and Control PIN/PDC RobuStore Source: Andrew Chien, UCSD

PI Larry Smarr Announced January 17, 2006 $24.5M Over Seven Years

Marine Genome Sequencing Project Measuring the Genetic Diversity of Ocean Microbes CAMERAs Sorcerer II Data Will Double Number of Proteins in GenBank!

Announced January 17, 2006

Flat File Server Farm W E B PORTAL Traditional User Response Request Dedicated Compute Farm (1000 CPUs) TeraGrid: Cyberinfrastructure Backplane (scheduled activities, e.g. all by all comparison) (10000s of CPUs) Web (other service) Local Cluster Local Environment Direct Access Lambda Cnxns Data- Base Farm 10 GigE Fabric CAMERAs Direct Access Core Architecture Will Create Next Generation Metagenomics Server Source: Phil Papadopoulos, SDSC, Calit2 + Web Services Sargasso Sea Data Sorcerer II Expedition (GOS) JGI Community Sequencing Project Moore Marine Microbial Project NASA Goddard Satellite Data Community Microbial Metagenomics Data

The Future Home of the Moore Foundation Funded Marine Microbial Ecology Metagenomics Complex First Implementation of the CAMERA Complex Photo Courtesy Joe Keefe, Calit2 Major Buildout of Calit2 Server Room Underway

Calit2 and the Venter Institute Will Combine Telepresence with Remote Interactive Analysis OptIPuter Visualized Data HDTV Over Lambda Live Demonstration of 21st Century National-Scale Team Science 25 Miles Venter Institute

UIC/UCSD 10GE CAVEWave on the National LambdaRail Emerging OptIPortal Sites CAVEWave Connects Chicago to Seattle to San Diego…and Washington D.C. as of 4/1/06 and JCVI as of 5/15/06 NEW! SunLight CICESE UW JCVI MIT SIO UCSD SDSU UIC EVL UCI OptIPortals

September 26-30, 2005 University of California, San Diego California Institute for Telecommunications and Information Technology Borderless Collaboration Between Global University Research Centers at 10Gbps i Grid 2005 T H E G L O B A L L A M B D A I N T E G R A T E D F A C I L I T Y Maxine Brown, Tom DeFanti, Co-Chairs 100Gb of Bandwidth into the Building More than 150Gb GLIF Transoceanic Bandwidth! 450 Attendees, 130 Participating Organizations 20 Countries Driving 49 Demonstrations 1- or 10- Gbps Per Demo

CineGrid Leverages OptIPuter Cyberinfrastructure to Enable Global Extreme Media Collaboration CineGrid Experiments Aim to Push the State of the Art in: –Streaming & Store-and-Forward File Transfer –Using High-speed, Low-latency Network Protocols –HDTV in Various Formats for: –Teleconferencing –Telepresence –Production –2K And 4K Digital Cinema Workflows and Distribution –Stereo –In High Resolution (2K, 4K) –Virtual Reality In Higher Resolution (24 Megapixel) –Distributed Tiled Displays With Megapixels –Long term Digital Archiving International Workshop: – Tokyo July 2006 – Calit2 Dec 2006 Source: Tom DeFanti, Laurin Herr

OptIPuter 4K Telepresence over IP at iGrid 2005 Demonstrated Technical Basis for CineGrid Keio University President Anzai UCSD Chancellor Fox Lays Technical Basis for Global Digital Cinema Sony NTT SGI New Calit2 Digital Cinema Auditorium

Calit2 Works with CENIC to Provide the California Optical Core for CineGrid Calit2 UCI USC SFSU UCB Plus, 1Gb and 10Gb Connections to: Seattle, Canada, Japan, Asia, Australia, New Zealand Chicago, Canada, Japan, Europe, Russia, China Tijuana CineGrid TM will Link UCSD/Calit2 and USC School of Cinema TV with Keio University Research Institute for Digital Media and Content Extended SoCal OptIPuter to USC School of Cinema- Television Calit2 UCSD Prototype of CineGrid TM Digital Archive of Films Partnering with SFSUs Institute for Next Generation Internet Source: Laurin Herr, Pacific Interface CineGrid TM Project Leader

Calit2 and the Venter Institute Test CineGrid with HDTV Movie by John Carter Live Demonstration of 21st Century Entertainment Delivery June 14, 2006 JCVI Sony HDTV JH-3 JC Venter Institute Rockville, MD Calit2 Auditorium StarLight Chicago

iGrid 2005 Kyoto Nijo Castle Source: Toppan Printing Interactive VR Streamed Live from Tokyo to Calit2 Over Dedicated GigE and Projected at 4k Resolution

iGrid 2005 Cultural Heritage China and USA Great Wall Cultural Heritage International Media Centre, China San Diego State University, USA Great Wall Society, China San Diego Supercomputer Center, USA Chinese Academy of Sciences, China GLORIAD, USA Chinese Institute of Surveying and Mapping, China Cybermapping Lab, University of Texas-Dallas, USA GEON viz/3D-scanning lab, University of Idaho, USA Stanford University, USA Data Acquisition from Laser Scanning Combined with Photogrammetry Enables the Construction of Unique Cultural Heritage Images from China. 3D Designs Of The Great Wall Are Combined With 3D Scans Of Physical Images And Satellite Imagery Stored On Servers In China And San Diego. Source : Maxine Brown, EVL UIC

NSFs Ocean Observatories Initiative (OOI) Envisions Global, Regional, and Coastal Scales LEO15 Inset Courtesy of Rutgers University, Institute of Marine and Coastal Sciences $300M in Presidents Budget for OOI

Coupling Regional and Coastal Ocean Observatories Using OptIPuter and Web/Grid Services LOOKING: (Laboratory for the Ocean Observatory Knowledge Integration Grid) LOOKING Funded by NSF ITR- John Delaney, UWash, PI)

Using the OptIPuter to Couple Data Assimilation Models to Remote Data Sources Including Biology Regional Ocean Modeling System (ROMS) NASA MODIS Mean Primary Productivity for April 2001 in California Current System

Interactive Remote Data and Visualization Services Multiple Scalable Displays Hardware Pixel Streaming Distributed Collaboration Scientific-Info Visualization AMR Volume Visualization Glyph and Feature Vis Visualization Services Data Mining for Areas of Interest Analysis and Feature Extraction Data Mining Services NCSA Altix Data and Vis Server An SDSC/NCSA Data CollaborationNational Laboratory for Advanced Data Research Linking to OptIPuter

The Synergy of Digital Art and Science Visualization of JPL Simulation of Monterey Bay Source: Donna Cox, Robert Patterson, NCSA Funded by NSF LOOKING Grant 4k Resolution

First Remote Interactive High Definition Video Exploration of Deep Sea Vents-Prototyping NEPTUNE Source John Delaney & Deborah Kelley, UWash Canadian-U.S. Collaboration

High Definition Still Frame of Hydrothermal Vent Ecology 2.3 Km Deep White Filamentous Bacteria on 'Pill Bug' Outer Carapace 1 cm. Source: John Delaney and Research Channel, U Washington