Presenter: Christian Charette, John Tkaczewski Duration: 30 minutes May Webinar: 10Gbps transfers.

Slides:



Advertisements
Similar presentations
Chapter 3: Introduction to Data Communications and Networking
Advertisements

FileCatalyst Performance Presentation.
TCP/IP MODEL Maninder Kaur
Transport Layer3-1 Transport Overview and UDP. Transport Layer3-2 Goals r Understand transport services m Multiplexing and Demultiplexing m Reliable data.
Introduction 1-1 Chapter 3 Transport Layer Intro and Multiplexing Computer Networking: A Top Down Approach 6 th edition Jim Kurose, Keith Ross Addison-Wesley.
August 10, Circuit TCP (CTCP) Helali Bhuiyan
Page 1 Dorado 400 Series Server Club Page 2 First member of the Dorado family based on the Next Generation architecture Employs Intel 64 Xeon Dual.
UKERNA CDI Trial Infrastructure for Content Delivery Steve Williams University of Wales Swansea.
Chapter 15 – Part 2 Networks The Internal Operating System The Architecture of Computer Hardware and Systems Software: An Information Technology Approach.
Webinar: Questions and Answers? July 17, Introduction John Tkaczewski, President and Co-Founder Check out our blog:
UMass DieselNet: A Disruption-Tolerant Network Testbed John Burgess Department of Computer Science UMass Amherst John Burgess Department of Computer Science.
Barracuda Networks Confidential1 Barracuda Backup Service Integrated Local & Offsite Data Backup.
Introduction to FileCatalyst Workflow May 21, 2015 Webinar at 11:00AM ET.
Virtual Network Servers. What is a Server? 1. A software application that provides a specific one or more services to other computers  Example: Apache.
© 2005,2006 NeoAccel Inc. Training Access Modes. © 2005,2006 NeoAccel Inc. Agenda 2. Access Terminals 6. Quick Access Terminal Client 3. SSL VPN-Plus.
CLIENT A client is an application or system that accesses a service made available by a server. applicationserver.
Programming mobile devices Part II Programming Symbian devices with Symbian C++
TCP/IP protocols Communication over Internet is mostly TCP/IP (Transmission Control Protocol over Internet Protocol) TCP/IP "stack" is software which allows.
The Network Layer. Network Projects Must utilize sockets programming –Client and Server –Any platform Please submit one page proposal Can work individually.
Review: – computer networks – topology: pair-wise connection, point-to-point networks and broadcast networks – switching techniques packet switching and.
IRODS performance test and SRB system at KEK Yoshimi KEK Building data grids with iRODS 27 May 2008.
Presenter: John Tkaczewski Duration: 30 minutes February Webinar: The Basics of Remote Data Replication.
Bonrix SMPP Client. Index Introduction Software and Hardware Requirements Architecture Set Up Installation HTTP API Features Screen-shots.
VirtualBox What you need to know to build a Virtual Machine.
Outline Overview Video Format Conversion Connection with An authentication Streaming media Transferring media.
Optimizing UDP-based Protocol Implementations Yunhong Gu and Robert L. Grossman Presenter: Michal Sabala National Center for Data Mining.
Sockets process sends/receives messages to/from its socket
RGEC MEERUT(IWT CS703) 1 Java Networking RGEC Meerut.
The Microsoft Technical Roadshow 2007 Rich Client Development in XAML Mark Johnston Developer & Platform Group Microsoft Ltd
Data transfer over the wide area network with a large round trip time H. Matsunaga, T. Isobe, T. Mashimo, H. Sakamoto, I. Ueda International Center for.
The Transport Layer application transport network data link physical application transport network data link physical application transport network data.
A Measurement Based Memory Performance Evaluation of High Throughput Servers Garba Isa Yau Department of Computer Engineering King Fahd University of Petroleum.
Chapter 15 – Part 2 Networks The Internal Operating System The Architecture of Computer Hardware and Systems Software: An Information Technology Approach.
High-speed TCP  FAST TCP: motivation, architecture, algorithms, performance (by Cheng Jin, David X. Wei and Steven H. Low)  Modifying TCP's Congestion.
DYNES Storage Infrastructure Artur Barczyk California Institute of Technology LHCOPN Meeting Geneva, October 07, 2010.
Introduction to FileCatalyst C++ API July 21st, 2015 Webinar at 11:00AM ET.
2: Application Layer1 Chapter 2 Application Layer Computer Networking: A Top Down Approach Featuring the Internet, 3 rd edition. Jim Kurose, Keith Ross.
Introduction to Computer Operating Systems
Installation of Storage Foundation for Windows High Availability 5.1 SP2 1 Daniel Schnack Principle Technical Support Engineer.
Enabling Technologies for Distributed Computing Dr. Sanjay P. Ahuja, Ph.D. Fidelity National Financial Distinguished Professor of CIS School of Computing,
Introduction TO Network Administration
Renesas Electronics America Inc. © 2010 Renesas Electronics America Inc. All rights reserved. Overview of Ethernet Networking A Rev /31/2011.
Webinar: Introducing TransferAgent September 25, 2014.
G046 Lecture 04 Task C Briefing Notes Mr C Johnston ICT Teacher
TCP Traffic Characteristics—Deep buffer Switch
Final EU Review - 24/03/2004 DataTAG is a project funded by the European Commission under contract IST Richard Hughes-Jones The University of.
Multimedia Retrieval Architecture Electrical Communication Engineering, Indian Institute of Science, Bangalore – , India Multimedia Retrieval Architecture.
Virtual Server Server Self Service Center (S3C) JI July.
Etere Video Assist The last step for a complete tapeless workflow Presented by: Emanuele Porfiri.
CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT Operating Systems.
Chapter 3 Getting Started. Copyright © 2005 Pearson Addison-Wesley. All rights reserved. Objectives To give an overview of the structure of a contemporary.
Presenter: John Tkaczewski Duration: 30 minutes March Webinar: Solving the problems of large attachments (& other web-based file transfer tools)
Introduction to Networks
January Webinar: How to Integrate FileCatalyst Java Applets
Node.Js Server Side Javascript
FileCatalyst Performance
Operating System.
Chapter 5 Network and Transport Layers
FileCatalyst – Partner Spotlight
Introduction to Networks
Xen Summit Spring 2007 Platform Virtualization with XenEnterprise
Chapter 5 Network and Transport Layers
Node.Js Server Side Javascript
April Webinar: Advanced Configuration of Order Forms in Workflow
File Transfer Issues with TCP Acceleration with FileCatalyst
Chapter 15 – Part 2 Networks The Internal Operating System
Windows Virtual PC / Hyper-V
Beyond FTP & hard drives: Accelerating LAN file transfers
News flash: Unlimi-Tech wins a Emmy
Exceptions and networking
Presentation transcript:

Presenter: Christian Charette, John Tkaczewski Duration: 30 minutes May Webinar: 10Gbps transfers

Presenter Introduction | © Copyright FileCatalyst, John Tkaczewski Co-Founder and President of Unlimi-Tech Software Oversees creative and technical vision for FileCatalyst Webmail and FileCatalyst Workflow Blog: Christian Charette Director of Software Development Project lead on 10 Gbps research

Agenda FileCatalyst Direct Overview Quick refresher why TCP doesn’t work in high speed networks 10Gbps test infrastructure – Ie: what are we using to perform the demo? Watch the data fly! Questions (please type them in as we go) | © Copyright FileCatalyst,

FileCatalyst Direct Overview | © Copyright FileCatalyst, Software architecture is Client-Server. One side acts as the main file repository (FileCatalyst Server) One or many clients connect to server, and push/pull data Clients include: – automated interfaces (API, CLI, Javascript, C++) – web components (applets) – feature rich applications (HotFolder, Express, mobile) Primarily written in Java, works on every major platform (Windows, Linux, Mac OSX, iOS, Android, and more) Transport protocol utilizes proprietary UDP, with error correction and congestion control built into the application layer (instead of relying on TCP network layer)

Putting 10 Gbps in context | © Copyright FileCatalyst, Gbps transfers an average BlueRay disk (26GB) in under 30 seconds Cost of 10 Gbps dedicated lines is dropping – becoming cost effective to many companies (~10K/month US to UK) 1 Gbps fiber connections becoming available to residential consumers

TCP vs UDP for packet loss & latency | © Copyright FileCatalyst, Quick refresher: TCP can’t cut it. Data provided as part of Photonics North 2012 presentation, FTP data uses FileZilla client connecting to PureFTPDPhotonics North 2012

TCP vs UDP for packet loss & latency | © Copyright FileCatalyst, TCP cannot handle packet loss -- even for networks with small latency. High latency networks fall apart under real world conditions.

TCP vs UDP for packet loss & latency | © Copyright FileCatalyst, You cannot transfer files at 10 Gbps with TCP. It just doesn’t work.

10 Gbps transfers | © Copyright FileCatalyst, Gbps research project started in May Project completed 6 months later with Nov 2011 release of FileCatalyst Direct v3.0, able to transfer at full 10gbps speed. 1½ years of successful deployments of 2-10 Gbps in the field. Tested in 2012 by independent researchers (ex-CEO of competitor Tixel GmbH), validated as most stable transfer results between 5 managed transport solutions. Comparison of Contemporary Solutions for High Speed Data Transport on WAN 10 Gbit/s ConnectionsComparison of Contemporary Solutions for High Speed Data Transport on WAN 10 Gbit/s Connections, Dmitry Kachan, Eduard Siemens, Vyacheslav Shuvalov

What are we using to run this demo? | © Copyright FileCatalyst, FileCatalyst HotFolder Ares x.x.x.58 FileCatalyst Server Hermes x.x.x.59

What are we using to run these tests? | © Copyright FileCatalyst, Machines utilize standard components circa Each machine costs approx. $5000 to put together. – Intel Westmere 5600 series (used in demo testbed) – AMD Magny-Cours 6100 Opterons FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2

What are we using to run these tests? | © Copyright FileCatalyst, Machine has 48GB total, 36GB reserved for RAMFS (so 12GB for OS + FileCatalyst). For the purpose of these tests, going to read/write to RAM drive. Why? Want to remove disks as part of the equation for demo to simplify things (everybody has different IO solutions in place). If your disks can handle the speeds, our software can transfer it. FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2

What are we using to run these tests? | © Copyright FileCatalyst, In demo, to simulate WAN environment, a Linktropy box from Apposite Technologies is used – Control link speed, packet loss, and link latency FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 WAN Emulator Apposite Technologies

So what are we going to see? | © Copyright FileCatalyst, FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 WAN Emulator Apposite Technologies

So what are we going to see? | © Copyright FileCatalyst, (1) Going to test most basic scenario: – Transfer static file from point A to B – 30GB file… slightly larger than the average Blue Ray disk. FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 WAN Emulator Apposite Technologies

So what are we going to see? | © Copyright FileCatalyst, (2) Going to test standard live media scenario: – Growth files (5) simulate live video feeds saved to system – Move files in real-time to destination as files are output FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu LTS Java bit FileCatalyst v3.2 WAN Emulator Apposite Technologies

What are we using to run these tests? | © Copyright FileCatalyst, Configurations used for single file 10gbps: – Server & HotFolder each using 4GB of RAM – MTU 9000 (packet 8972 bytes) – Blocks: 10MB – Number of sender threads: 50 – Sender sockets: 5 – Receiver sockets: 3 – RTT congestion control (because test scenario utilizes packet loss) – Congestion control set to 7 (more aggressive) Linktropy setup to simulate LA -> NY transfers – Latency of 60 ms – 0.1% packet loss

DEMOS | © Copyright FileCatalyst,

Upcoming Events: - NYC Digital Asset Managers, (May 29 th ) - Broadcast Asia, Singapore (June 18-21) - IBC, Amsterdam (Sept 13-17) - Interop New York (Oct 2-3) - Broadcast India, Mumbai (Oct 9-11) And of course don’t forget our monthly webinar....

Thank you for attending our May webinar! Questions? Please type in your questions.