Logistical Networking: Buffering in the Network Prof. Martin Swany, Ph.D. Department of Computer and Information Sciences.

Slides:



Advertisements
Similar presentations
Recent Developments in Logistical Networking Micah Beck, Assoc. Prof. & Director Logistical Computing & Internetworking (LoCI) Lab Computer Science Department.
Advertisements

High Performance Computing Course Notes Grid Computing.
CSE 6590 Department of Computer Science & Engineering York University 1 Introduction to Wireless Ad-hoc Networking 5/4/2015 2:17 PM.
GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.
Computer Networks Fall, 2007 Prof Peterson. CIS 235: Networks Fall, 2007 Western State College  What is “store and forward”?  What is a buffer / queue?
Chris Blondia Ne X tworking’03 June 23-25,2003, Chania, Crete, Greece The First COST-IST(EU)-NSF(USA) Workshop on EXCHANGES & TRENDS IN N ETWORKING 1 Layer.
Astrophysics, Biology, Climate, Combustion, Fusion, Nanoscience Working Group on Simulation-Driven Applications 10 CS, 10 Sim, 1 VR.
Grids and Grid Technologies for Wide-Area Distributed Computing Mark Baker, Rajkumar Buyya and Domenico Laforenza.
CS335 Networking & Network Administration Tuesday, April 20, 2010.
The Difficulties of Distributed Data Douglas Thain Condor Project University of Wisconsin
Simo Niskala Teemu Pasanen
Module – 7 network-attached storage (NAS)
Module I Overview of Computer Architecture and Organization.
UNIVERSITY of MARYLAND GLOBAL LAND COVER FACILITY High Performance Computing in Support of Geospatial Information Discovery and Mining Joseph JaJa Institute.
Infiniband enables scalable Real Application Clusters – Update Spring 2008 Sumanta Chatterjee, Oracle Richard Frank, Oracle.
 Sergey Barskiy  Principal consultant at Magenic Technologies  
Chapter 9.
Open Science Grid For CI-Days Internet2: Fall Member Meeting, 2007 John McGee – OSG Engagement Manager Renaissance Computing Institute.
CS An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu.
Big Data: Movement, Crunching, and Sharing Guy Almes, Academy for Advanced Telecommunications 13 February 2015.
Sponsored by the National Science Foundation Research & Experiments on GENI GENI CC-NIE Workshop NSF Mark Berman, Mike Zink January 7,
Local IBP Use at Vanderbilt University Advanced Computing Center for Research and Education.
1 High-Level Carrier Requirements for Cross Layer Optimization Dave McDysan Verizon.
Hub  A hub is a piece of hardware that connects multiple wired devices together on a single network. Hubs work at the physical layer (layer 1) of the.
The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Dataset Caitlin Minteer & Kelly Clynes.
Open Science Grid For CI-Days Elizabeth City State University Jan-2008 John McGee – OSG Engagement Manager Manager, Cyberinfrastructure.
The Singapore Advanced Research & Education Network.
RNP's Digital Video Working Group Applying Logistical Networking to Content Delivery.
GENI Experiments in Optimizing Network Environments using XSP Ezra Kissel and Martin Swany University of Delaware Abstract Our proposal is to build, deploy.
Working together towards a next generation storage element Surya D. Pathak Advanced Computing Center for Research and Education.
1 Logistical Computing and Internetworking: Middleware for the Use of Storage in Communication Micah Beck Jack Dongarra Terry Moore James Plank University.
Opportunities in Parallel I/O for Scientific Data Management Rajeev Thakur and Rob Ross Mathematics and Computer Science Division Argonne National Laboratory.
Logistical Networking Micah Beck, Research Assoc. Professor Director, Logistical Computing & Internetworking (LoCI) Lab Computer.
Ethernet Basics – 8 Routers. Routers and Routing Definition of a router- A device which provides a path from a node on one network or subnet to a node.
Network Essentials -III Prof. Sujata Rao Lesson 12.
An Exposed Approach to Reliable Multicast in Heterogeneous Logistical Networks Micah Beck, Assoc. Prof. & Director Logistical Computing & Internetworking.
ASCR/ESnet Network Requirements an Internet2 Perspective 2009 ASCR/ESnet Network Requirements Workshop April 15/16, 2009 Richard Carlson -- Internet2.
1 Mobile Management of Network Files Alex BassiMicah Beck Terry Moore Computer Science Department University of Tennessee.
George Kola Computer Sciences Department University of Wisconsin-Madison DiskRouter: A Mechanism for High.
Wide Area Data Sharing with Logistical Networking Micah Beck, Assoc. Prof. & Director Logistical Computing & Internetworking (LoCI) Lab Computer Science.
F. Douglas Swesty, DOE Office of Science Data Management Workshop, SLAC March Data Management Needs for Nuclear-Astrophysical Simulation at the Ultrascale.
The Data Logistics Toolkit Martin Swany Professor, School of Informatics and Computing Executive Associate Director, Center for Research in Extreme Scale.
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 1 Enabling Supernova Computations by Integrated Transport and Provisioning Methods Optimized.
Department of Computing, School of Electrical Engineering and Computer Sciences, NUST - Islamabad KTH Applied Information Security Lab Secure Sharding.
AMQP, Message Broker Babu Ram Dawadi. overview Why MOM architecture? Messaging broker like RabbitMQ in brief RabbitMQ AMQP – What is it ?
Network Components Basics!. Network HUB  Used to connect multiple Ethernet devices together  Layer 1 of the OSI model  Not used much today.
An End-to-End Approach to Scalable Network Storage Micah Beck, Associate Professor Director, Logistical Computing & Internetworking (LoCI) Lab Terry Moore,
Simple Infrastructure to Exploit 100G Wide Are Networks for Data-Intensive Science Shawn McKee / University of Michigan Supercomputing 2015 Austin, Texas.
Challenges in the Next Generation Internet Xin Yuan Department of Computer Science Florida State University
Computer Network Architecture Lecture 6: OSI Model Layers Examples 1 20/12/2012.
An Architectural Approach to Managing Data in Transit Micah Beck Director & Associate Professor Logistical Computing and Internetworking Lab Computer Science.
Company LOGO Network Management Architecture By Dr. Shadi Masadeh 1.
NETWORK DEVICES Department of CE/IT.
09/13/04 CDA 6506 Network Architecture and Client/Server Computing Peer-to-Peer Computing and Content Distribution Networks by Zornitza Genova Prodanoff.
August 28, 2003APAN, Logistical Networking WS DiDaS Distributed Data Storage Ludek Matyska Masaryk University, Institute of Comp. Sci. and CESNET, z.s.p.o.
1 Data Management for Internet Backplane Protocol by Tang Ming Assoc/Prof. Francis Lee School of Computer Engineering, Nanyang Technological University,
SDN-SF LANL Tasks. LANL Research Tasks Explore parallel file system networking (e.g. LNet peer credits) in order to give preferential treatment to isolated.
ORNL Site Report ESCC Feb 25, 2014 Susan Hicks. 2 Optical Upgrades.
Fault – Tolerant Distributed Multimedia Streaming Web Application By Nirvan Sagar – Srishti Ganjoo – Syed Shahbaaz Safir
Landsat Remote Sensing Workflow
WP18, High-speed data recording Krzysztof Wrona, European XFEL
CMS and Logistical Storage
Grid Computing.
Storage Virtualization
Distributed Storage in Automated Data Transfer
TASK 4 Guideline.
AmericaView REDDnet? Geospatial Data Delivery P.R. Blackwell
Outline Problem DiskRouter Overview Details Real life DiskRouters
Department of Computer Science University of York
Scheduled Accomplishments
Presentation transcript:

Logistical Networking: Buffering in the Network Prof. Martin Swany, Ph.D. Department of Computer and Information Sciences

Network Logistics and Buffering Logistics is time-related positioning of resources Buffers are temporary storage areas used when transferring data Storage has a continuum of persistence and durability Network devices have some buffer capacity High-performance I/O depends on buffering at various levels

Logistical Networking Logistical Networking improves data movement performance and functionality via buffering across a range of parameters and layers Logistical Session Layer / Phoebus Plentiful, highly transitory, moderate capacity Buffer duration from milliseconds to minutes Implements “Store and Forward” along the end-to-end path for improved throughput Internet Backplane Protocol / REDDnet Less plentiful, more persistent, significant capacity Buffer duration from minutes to months Implements “Working Storage” used in general distributed applications

DCN End-to-End Session

Internet2 Deployment (in progress) Phoebus nodes at router POPs Internet2 Phoebus Gateway

Cluster Supercomputer depots End User Producer Consumer

7

Integrated LN Suite Together, Phoebus and IBP implement integrated buffering at all scales Wide range of performance improvement and functionality for scalable I/O and data movement Pipelining Caching

Users, Collaborators, Support Users include Terascale Supernova Initiative, DOE Fusion, CMS, LIGO, AmericaView, National Geospatial Data Archive, Library of Congress, … Support from the NSF, DOE, Library of Congress, Internet2 University of Delaware, University of Tennessee, Vanderbilt University, Stephen F. Austin State University

End Thank you for your attention Questions?