1. Application-Network Tracing and Correlation in Datacenters (ANTACID)
Plan of Action
• Collect packet traces + application logs
– MapReduce as case study
– Run at Yahoo! If possible, else on EC2
• Correlate low level and high level logs
– What do app logs tell us about network?
–  How network affects app performance?
• Leverage Chukwa data collection tool
–  Collection of distributed packet-traces
– Centralized + scalable storage on HDFS
– Simplifies analysis and visualization
A. Rabkin & A. Konwinski, UC Berkeley
Impact
• Simple tools for collecting and analyzing large network level traces
will be rapidly adopted by many
corporations, such as Yahoo!.
• Collection, organization, and storage of low level trace data will facilitate work by other researchers.
• Availability of data and analysis tools (through Chukwa) will facilitate new uses of the data, e.g. replayable traces.
Schedule
• Current: Matrix visualization of data collected on node Hadoop cluster.
• Oct 10: Collect packet traces on 7 node cluster
• Oct 23: Discuss Chukwa integration at CCA
• Nov 4: Collect large traces and logs
• Dec 9: Project poster