Noise Can Help: Accurate and Efficient Per-flow Latency Measurement without Packet Probing and Time Stamping Dept. of Computer Science and Engineering.

Slides:

Advertisements

Similar presentations

Author: Chengchen, Bin Liu Publisher: International Conference on Computational Science and Engineering Presenter: Yun-Yan Chang Date: 2012/04/18 1.

Advertisements

Analysis of : Operator Scheduling in a Data Stream Manager CS561 – Advanced Database Systems By Eric Bloom.

A Search Memory Substrate for High Throughput and Low Power Packet Processing Sangyeun Cho, Michel Hanna and Rami Melhem Dept. of Computer Science University.

Florin Dinu T. S. Eugene Ng Rice University Inferring a Network Congestion Map with Traffic Overhead 0 zero.

A Scalable and Reconfigurable Search Memory Substrate for High Throughput Packet Processing Sangyeun Cho and Rami Melhem Dept. of Computer Science University.

Data Streaming Algorithms for Accurate and Efficient Measurement of Traffic and Flow Matrices Qi Zhao*, Abhishek Kumar*, Jia Wang + and Jun (Jim) Xu* *College.

Fine-Grained Latency and Loss Measurements in the Presence of Reordering Myungjin Lee, Sharon Goldberg, Ramana Rao Kompella, George Varghese.

Estimating TCP Latency Approximately with Passive Measurements Sriharsha Gangam, Jaideep Chandrashekar, Ítalo Cunha, Jim Kurose.

Enabling Flow-level Latency Measurements across Routers in Data Centers Parmjeet Singh, Myungjin Lee Sagar Kumar, Ramana Rao Kompella.

Active Queue Management: Theory, Experiment and Implementation Vishal Misra Dept. of Computer Science Columbia University in the City of New York.

Every Bit Counts – Fast and Scalable RFID Estimation Muhammad Shahzad and Alex X. Liu Dept. of Computer Science and Engineering Michigan State University.

Copyright © 2005 Department of Computer Science 1 Solving the TCP-incast Problem with Application-Level Scheduling Maxim Podlesny, University of Waterloo.

Secure Unlocking of Mobile Touch Screen Devices by Simple Gestures – You can see it but you can not do it Arjmand Samuel Microsoft Research Muhammad Shahzad.

PERSISTENT DROPPING: An Efficient Control of Traffic Aggregates Hani JamjoomKang G. Shin Electrical Engineering & Computer Science UNIVERSITY OF MICHIGAN,

1 Sensor Relocation in Mobile Sensor Networks Guiling Wang, Guohong Cao, Tom La Porta, and Wensheng Zhang Department of Computer Science & Engineering.

Receiver-driven Layered Multicast S. McCanne, V. Jacobsen and M. Vetterli SIGCOMM 1996.

The War Between Mice and Elephants Presented By Eric Wang Liang Guo and Ibrahim Matta Boston University ICNP

1 Ossama Younis and Sonia Fahmy Department of Computer Sciences Purdue University For slides, technical report, and implementation, please see:

Reverse Hashing for High-speed Network Monitoring: Algorithms, Evaluation, and Applications Robert Schweller 1, Zhichun Li 1, Yan Chen 1, Yan Gao 1, Ashish.

Computer Science 1 Providing QoS through Active Domain Management Liang Guo, Ibrahim Matta Quality-of-Service Networking Lab CS Department Boston University.

A Practical Packet Reordering Mechanism with Flow Granularity for Parallel Exploiting in Network Processors 13 th WPDRTS April 4, 2005 Beibei Wu, Yang.

SafeQ: Secure and Efficient Query Processing in Sensor Networks Fei Chen and Alex X. Liu Department of Computer Science and Engineering Michigan State.

Reverse Hashing for Sketch Based Change Detection in High Speed Networks Ashish Gupta Elliot Parsons with Robert Schweller, Theory Group Advisor: Yan Chen.

Scheduling with Optimized Communication for Time-Triggered Embedded Systems Slide 1 Scheduling with Optimized Communication for Time-Triggered Embedded.

Performance Evaluation

SACRIO - An Active Buffer Mangement Scheme for Differentiaed Services Networks Saikrishnan Gopalakrishnan Cisco Systems Narasimha Reddy Texas A & M University.

User-level Internet Path Diagnosis R. Mahajan, N. Spring, D. Wetherall and T. Anderson.

RFID Cardinality Estimation with Blocker Tags

OS Fall ’ 02 Performance Evaluation Operating Systems Fall 2002.

Not All Microseconds are Equal: Fine-Grained Per-Flow Measurements with Reference Latency Interpolation Myungjin Lee †, Nick Duffield‡, Ramana Rao Kompella†

1 Enabling Large Scale Network Simulation with 100 Million Nodes using Grid Infrastructure Hiroyuki Ohsaki Graduate School of Information Sci. & Tech.

SIGCOMM 2002 New Directions in Traffic Measurement and Accounting Focusing on the Elephants, Ignoring the Mice Cristian Estan and George Varghese University.

Scalable and Efficient Data Streaming Algorithms for Detecting Common Content in Internet Traffic Minho Sung Networking & Telecommunications Group College.

Mar 1, 2004 Multi-path Routing CSE 525 Course Presentation Dhanashri Kelkar Department of Computer Science and Engineering OGI School of Science and Engineering.

Multiple Aggregations Over Data Streams Rui ZhangNational Univ. of Singapore Nick KoudasUniv. of Toronto Beng Chin OoiNational Univ. of Singapore Divesh.

Probabilistic Optimal Tree Hopping for RFID Identification Muhammad Shahzad Alex X. Liu Dept. of Computer Science and Engineering Michigan State University.

CS848 Class Project: A Survey on QoS for Multi-tier Web Systems Huaning(Mike) Nie

High-Fidelity Latency Measurements in Low-Latency Networks Ramana Rao Kompella Myungjin Lee (Purdue), Nick Duffield (AT&T Labs – Research)

A Passive Approach to Sensor Network Localization Rahul Biswas and Sebastian Thrun International Conference on Intelligent Robots and Systems 2004 Presented.

1 Fast packet classification for two-dimensional conflict-free filters Department of Computer Science and Information Engineering National Cheng Kung University,

Efficient Cache Structures of IP Routers to Provide Policy-Based Services Graduate School of Engineering Osaka City University

PROTEUS: Network Performance Forecast for Real- Time, Interactive Mobile Applications Qiang Xu* Sanjeev Mehrotra# Z. Morley Mao* Jin Li# *University of.

Secure Unlocking of Mobile Touch Screen Devices by Simple Gestures – You can see it but you can not do it Muhammad Shahzad, Alex X. Liu Michigan State.

Muhammad Shahzad Alex X. Liu Dept. of Computer Science and Engineering

An Efficient Gigabit Ethernet Switch Model for Large-Scale Simulation Dong (Kevin) Jin.

A N I N - MEMORY F RAMEWORK FOR E XTENDED M AP R EDUCE 2011 Third IEEE International Conference on Coud Computing Technology and Science.

Emir Halepovic, Jeffrey Pang, Oliver Spatscheck AT&T Labs - Research

An Efficient Gigabit Ethernet Switch Model for Large-Scale Simulation Dong (Kevin) Jin.

1 CMP-MSI.07 CARES/SNU A Reusability-Aware Cache Memory Sharing Technique for High Performance CMPs with Private Caches Sungjune Youn, Hyunhee Kim and.

Noise Can Help: Accurate and Efficient Per-flow Latency Measurement without Packet Probing and Time Stamping Michigan State University SIGMETRICS 14.

Block-Based Packet Buffer with Deterministic Packet Departures Hao Wang and Bill Lin University of California, San Diego HSPR 2010, Dallas.

1 Monitoring: from research to operations Christophe Diot and the IP Sprintlabs ipmon.sprintlabs.com.

1 Scalability and Accuracy in a Large-Scale Network Emulator Nov. 12, 2003 Byung-Gon Chun.

A Stochastic Frame Based Approach to RFID Tag Searching Ann L. Wang Dept. of Computer Science and Engineering Michigan State University Joint work with.

SketchVisor: Robust Network Measurement for Software Packet Processing

AMI to SmartGrid “DATA”

Simulation and Exploration of

Updating SF-Tree Speaker: Ho Wai Shing.

Language-Directed Hardware Design for Network Performance Monitoring

Pyramid Sketch: a Sketch Framework

Qun Huang, Patrick P. C. Lee, Yungang Bao

Providing QoS through Active Domain Management

ECEN “Internet Protocols and Modeling”

Congestion Control in SDN-Enabled Networks

Modular Edge-connected data centers

Congestion Control in SDN-Enabled Networks

Lu Tang , Qun Huang, Patrick P. C. Lee

Modular Edge-connected data centers

NitroSketch: Robust and General Sketch-based Monitoring in Software Switches Alan (Zaoxing) Liu Joint work with Ran Ben-Basat, Gil Einziger, Yaron Kassner,

Author: Ramana Rao Kompella, Kirill Levchenko, Alex C

Presentation transcript:

Noise Can Help: Accurate and Efficient Per-flow Latency Measurement without Packet Probing and Time Stamping Dept. of Computer Science and Engineering Michigan State University East Lansing, Michigan, 48824, USA Muhammad Shahzad Alex X. Liu

2 Latency Matters!  Applications ─ Financial trading ─ HPC  Architecture ─ CDNs ─ Data centers

“When considering how to reduce latency, the first step is to measure it.” (Joanne Kinsella, Head of Portfolio, British Telecom)

4 Flow 1 = 1ms Flow 2 = 1ms Flow 3 = 1.25ms Flow 4 = 4ms  Aggregate Latency Measurement ─ Measure average latency ─ Guaranteeing average ≠ Guaranteeing each  Per-flow Latency Measurement ─ Measure latency of each flow  Applications ─ ISP operators ─ ISP customers Types of Latency Measurements Aggregate Latency = 1.7ms

5 Prior Art and Limitations  Aggregate Latency Measurement ─ LDA [SIGCOMM’09] ─ FineComb [SIGMETRICS’11]  Per-flow Latency Measurement ─ RLI [SIGCOMM’10]: active probes ─ MAPLE [IMC’12]: timestamps  Commercial Solution ─ Corvil’s latency monitoring devices ─ USD 180,000 for a 2 × 10Gbps box

6 Problem Statement

7 Basic Idea

8 Recording Phase: a naïve solution  One counter per flow: 1-1 mapping  Problem ─ Overflow vs. Underutilization  Reason ─ 1-1 mapping: flows counters

9 Recording Phase: COLATE  Cost per packet: ─ One hash computation ─ One memory update

10 Querying Phase Latency of Flow ? Take average

11 Optimal Parameter Selection

12 Optimal Parameter Selection

13 Performance Evaluation  Network Traces  Simulated queue traversal to get departure timestamps ─ RED queue management strategy TraceDurationNo. of Packets No. of Flows CHIC5 mins37.3M3.01M ICSI41.1 hrs46.9M0.387M DC1.08 hrs19.9M0.439M

14 Accuracy

15 Comparison with RLI  Implemented RLI (SIGCOMM’10)

16 Advantages over Prior-Art  Proposed an accurate and efficient per-flow latency measurement scheme ─ Reliable ─ Passive ─ Scalable ─ Efficient: Memory and Computations ─ Flexible  More in the paper ─ Standard deviation in latencies of packets in a flow ─ Theoretical development

17 Questions?