1. Department of Computer Science Northwestern University
Inferring the Topology and Traffic Load of Parallel Programs in a VM environment
Ashish Gupta
Peter Dinda
Department of Computer Science
Northwestern University
Introduce the project.

2. Overview Motivation behind parallel programs in a VM environment
Goal: To infer the communication behavior
Offline implementation
Evaluating with parallel benchmarks
Online Monitoring in a VM environment
Conclusions
More specific

3. Virtuoso: A VM based abstraction for a Grid environment

4. Add: we have all this wonderful toys, environment where we can run parallel applications..
Lot of complexity.
Resources are out there , but increbvidly complicated :
Virtuso: provide a clean abtraction on top of all this

5. A distributed computing environment based on Virtual Machines
Motivation
A distributed computing environment based on Virtual Machines
Raw machines connected to user’s network
Our Focus: Middleware support to hide the Grid complexity
Virtuoso project.
Efficient execution of parallel applications means better performance, cost and resource utilization for both the user and the service providers

7. Can ssh into the machine
They can use the group of machines as if they would use a local cluster….
The middleware is repsonsible for making it look like a cluster

8. A distributed computing environment based on Virtual Machines
Motivation
A distributed computing environment based on Virtual Machines
Raw machines connected to user’s network
Our Focus: Middleware support to hide the Grid complexity
Our goal here: Efficient execution of Parallel applications in such an environment
Virtuoso project.
Efficient execution of parallel applications means better performance, cost and resource utilization for both the user and the service providers
Add: our goal

9. Intelligent Placement and of parallel applications
Application Behavior
Intelligent Placement and
virtual networking
of parallel applications
By combining the knowledge of these two aspects, we are in a position to make decisions.
Our decision making capability depends on two factors:
VM Encapsulation provide flexibility in placement and migration of parallel applications on the network
Virtual Networks : can create custom routing and topologies on the underlying physical network
The goal of this project focus on the upper right hand part: understanding parallel application communication behavior
So what does the behavior look like: Parallel Applications communicate and compute. Communicate according to certain topologies.
Also bandwidth
VM Encapsulation
Virtual Networks
With VNET

10. Abstraction: A set of VMs on same Layer 2 network Virtual Ethernet LAN
VNET
Abstraction: A set of VMs on same Layer 2 network
Virtual Ethernet LAN

11. An online topology inference framework for a VM environment
Goal of this project ?
Application Topology
Low Level Traffic
Monitoring
An online topology inference framework for a VM environment
Add: Application topology

12. Design an offline framework Evaluate with parallel benchmarks
Approach
Design an offline framework
Evaluate with parallel benchmarks
What is offline ?
To test hypothesis
Using tcpdump
If successful, design an online framework for VMs

13. An offline topology inference framework
Goal:
A test-bed for traffic monitoring and evaluating topology inference methods

14. The offline method Synced Parallel Traffic Monitoring
Traffic Filtering and Matrix Generation
Wrote a set of perl scripts for each step to automate the whole process. Manually can be painful.
Use tcpdump to capture traffic for each host
Then we can do the require filtering to remove any non-application projects
Do flow aggregation to generate a traffic matrix for the hosts
Add: the steps are simple, explain the filterig and algorithm here.
Matrix Analysis and Topology Characterization

15. The offline method Synced Parallel Traffic Monitoring
Traffic Filtering and Matrix Generation
Then after we get the traffic matrix, we need to have algorithms to infer the topology from the traffic matrix, leaving out the noise
For this there are many ways, like pruning, pattern detection etc.
Currently a simple linear scaling method is used to infer the topology. Explain the technique
Matrix Analysis and Topology Characterization

16. The offline method Synced Parallel Traffic Monitoring
Traffic Filtering and Matrix Generation
A simple visualization module was added which can visualize the inferred topology.
Using SAMBA from Georgia Tech. Example
Matrix Analysis and Topology Characterization

17. The offline method Synced Parallel Traffic Monitoring
Traffic Filtering and Matrix Generation
A simple visualization module was added which can visualize the inferred topology.
Using SAMBA from Georgia Tech. Example
Matrix Analysis and Topology Characterization

18. The offline method PVMPOV Inference Synced Parallel Traffic Monitoring
Traffic Filtering and Matrix Generation
A simple visualization module was added which can visualize the inferred topology.
Using SAMBA from Georgia Tech. Example
Matrix Analysis and Topology Characterization
PVMPOV Inference

19. Infer.pl Synced Parallel Traffic Monitoring
Traffic Filtering and Matrix Generation
Explain the visualization generated.
Finally a one step process:
Doitall [parallel program name] and it went through all the steps and displayed the infered topology
Add: pvmpov running
Matrix Analysis and Topology Characterization
Infer.pl

20. Parallel Benchmarks Evaluation
Goal:
To test the practicality of low level traffic based inference

21. Parallel Benchmarks used
Synthetic benchmarks: Patterns
N-dimensional mesh-neighbor
N-dimensional toroid-neighbor
N-dimensional hypercubes
Tree reduction
All-to-All
Scheduling mechanism to generate deadlock free and efficient schemes
The patterns application was custom - written
Used PVM: Parallel virtual Machine
A parallel application development library
Speak: Our implementation of these communication patterns if efficient and deadlock free 1 2 3

22. Application benchmarks
NAS PVM benchmarks
Popular benchmarks for parallel computing
5 benchmarks
PVM-POV : Distributed Ray Tracing
Many others possible…
The inference not PVM specific
Applicable to all communication .
e.g. MPI, even non-parallel apps
Computational and communication characteristics in computational aerodynamics

23. Patterns application 3-D Toroid 3-D Hypercube 2-D Mesh Reduction Tree
Add: identify the topologies
Reduction Tree
All-to-All

24. PVM NAS benchmarks
Parallel Integer Sort

25. Traffic Matrix for PVM IS benchmark
This traffic matrix shows how placement of host1 is crucial on the network. High bandwidth connectivity should be there
Also the virtual network should facilitate the communication between other hosts and host1
Also the decisions need to be dynamic since the properties of the physical network may change
Speak: which host talks to which host……

26. Traffic Matrix for PVM IS benchmark
This traffic matrix shows how placement of host1 is crucial on the network. High bandwidth connectivity should be there
Also the virtual network should facilitate the communication between other hosts and host1
Also the decisions need to be dynamic since the properties of the physical network may change
Speak: host1 is communication is intensive: we can see it sends a lot from tehc column, and receives a lot as from the row
Placement of host1 is crucial on the network

27. An Online Topology Inference Framework: VTTIF
Goal:
To automatically detect, monitor and report the global traffic matrix for a set of VMs running on a overlay network

28. VNET Overall Design Abstraction: A set of VMs on same Layer 2 network
Virtual Ethernet LAN
The framework can aggregate all traffic for all periods of time. Need to detect only interesting communication behavior so that appropriate action can be taken.
Add: what is vnet ?

29. A VNET virtual layer A Virtual LAN over wide area VNET Layer
Physical Layer

30. Extend VNET to include the required features The Challenge here
Overall Design
VNET
Abstraction: A set of VMs on same Layer 2 network
Extend VNET to include the required features
Monitoring at Ethernet packet level
The Challenge here
Lacks manual control
Detecting interesting parallel program communication ?
The framework can aggregate all traffic for all periods of time. Need to detect only interesting communication behavior so that appropriate action can be taken.
Add: what is vnet ?

31. Detecting interesting phenomenon
Reactive Mechanisms
Proactive Mechanisms
Like a Burglar Alarm
Video Surveillance
Certain address properties
Based on Traffic rate
Etc.
Provide support for queries by external agent
Rate based monitoring
Non-uniform discrete event sampling
What is the Traffic Matrix for the last n seconds ?
Explain both the mechanisms with examples

32. VM Network Scheduling Agent
Physical Host
VNET daemon
VNET overlay network
Traffic Analyzer
Rate based
Change detection
Traffic Matrix
Query Agent
To other VNET daemons
Add: text contrast is bad, text is small
VM Network Scheduling Agent

33. Traffic Matrix Aggregation
Each VNET daemon keeps track of local traffic matrix
Need to aggregate this information for a global view
When the rate falls, the local daemons push the traffic matrix (When do you push the traffic matrix ?)
Operation is associative: reduction trees for scalability
Scalability issues
Performance is ok in latency and bandwidth tests.
The proxy daemon

34. Used 4 Virtual Machines over VNET NAS IS benchmark
Evaluation
Used 4 Virtual Machines over VNET
NAS IS benchmark
We run the parallel program and the Framework automatically detects the communication and outputs the aggregated traffic matrix when the communication ends
The traffic matrix was much larger than that from the physical hosts. Reasons: packets dropped in tcpdump in the offline case.

35. Possible to infer the topology low level traffic monitoring
Conclusions
Possible to infer the topology
with
low level traffic monitoring
A Traffic Inference Framework for Virtual Machines
Ready to move on to future steps:
Adaptation for Performance
Add: current and future work….
Add: the numbers on adaptation
Add: questions/advertizing…

36. Capabilities for dynamic adaptation into VNET
Current Work
Capabilities for dynamic adaptation into VNET
Spatial Inference  Network Adaptation for Improved Performance
Prelim Results: Improved performance upto 40% in execution time
Looking into benefits of Dynamic Adaptation

37. plab.cs.northwestern.edu http://virtuoso.cs.northwestern.edu
For more information
VNET is available for download
PLAB web site:
plab.cs.northwestern.edu