1. Monkey See, Monkey Do A Tool for TCP Tracing and Replaying
Yu-Chung Cheng
Stefan Savage
Geoffrey M. Voelker
Dept. of Computer Science & Engineering
University of California, San Diego
Urs Hölzle
Neal Cardwell
11/16/2018
USENIX '04

2. Motivation Server benchmarks: the “what if” question Current solutions
Ex: how will changing the server in some way impact the client experience?
Current solutions
Synthetic workload testing: risk-free, convenient, not very realistic
Real user testing: realistic, risky, more hassle
Best of both worlds?
Realistic, convenient and risk-free
11/16/2018
USENIX '04

3. Goal Develop a tool, Monkey, that provides the best of both worlds
Monkey See: captures real traces, infers network and client parameters
Monkey Do: replays traces via emulation
Applied Monkey to Google workload
Captured Google search requests
Replayed and evaluated against Google servers
Evaluation
How realistic is Monkey’s replay?
Can Monkey answer the “what if” questions?
11/16/2018
USENIX '04

4. Monkey Design Monkey See Monkey Do Google Web Server Trace Capturing
TCP delay ACK policy, receiver buffer size
HTTP GET /search?q=Monkey
Client  server: delay 90 ms, 1% loss, 384 kbps, MTU 1500
Server  client: delay 90 ms, 0% loss, 128 kbps, MTU 1500
Extracting Trace Info
Client
Monkey See
Client Emulator
Network Emulator
Address Remapping 
Monkey Do
Server Emulator
Google Server kernel
LAN
11/16/2018
USENIX '04

5. Monkey See
Goal: capturing Google search traces and infer client/network parameters
Trace capture in front of server farm
Random connection sampling
Extracting info for each TCP connection
Search query from HTTP header
Path delays from RTT
Bandwidth by counting ACK spacing
Loss rate by counting retransmissions
Path MTU from TCP MSS option
Use of TCP delay ACKs by observing ACK sequences
11/16/2018
USENIX '04

6. Assumptions 3. Well-provisioned Google Web Server
1. Congestion/loss only happens on data path but not on ACK path
client
Google
Web server
Google DB
2. No congestion in Google intranet
TCP data packet
TCP ACK packet
11/16/2018
USENIX '04

7. Monkey Do
Goal: replay traces by emulating the client and network environment
Network emulator (dummynet)
Setup virtual dummynet pipes as data and ACK paths
Client emulator
Establishes connections to the server and send out search queries
Server emulator
Emulates the effect of caching on DB search delay
Runs on same kernel (Linux) as the real Google web daemon does
11/16/2018
USENIX '04

8. Replay in action: real Google search
Client
Web server
DB server
TCP Handshake
connect()
Static header
HTTP GET
write(HEADER)
HTTP 200 OK
DB Query delay
write(RESULT)
Search result
Time
11/16/2018
USENIX '04

9. Replay in action: replay version
Client Emulator
Server Emulator
TCP Handshake
connect()
Static header
HTTP GET
write(HEADER)
HTTP 200 OK
DB Query delay
write(RESULT)
Search result
Time
11/16/2018
USENIX '04

10. Evaluation Replay Validation Predictive Replay
How well does Monkey reproduce the workload?
Predictive Replay
How well does Monkey predict?
Metric: Google search response time
Time between the server receives the request and completes sending the result
Trace taken a weekday of Nov 2003
11/16/2018
USENIX '04

11. Replay Validation Experiment
How well does Monkey reproduce the workload?
Replay the workload without any changes and compare the results
Torig
Monkey See
Real Clients
Google
Server
Kernel
Web
Daemon
Treplay
Monkey Do
Monkey Client/ Network Emulator
Google
Server
Kernel
Monkey
Server Emulator
11/16/2018
USENIX '04

12. Replay Validation Result
% connections
Relative error of search response time [ms]
CDF of relative error in search response time per connection
Monkey tends to underestimate for long RTT connections
CDF of relative error in search response time per connection
86% of connections have response time within ±20% error
11/16/2018
USENIX '04

13. Monkey Client/ Network Emulator
Predictive Replay
Is Monkey’s replay predictive?
Compare replay result with result from real server changes
Experiment
Search response time vs. TCP initial congestion window size (cwin)
Tw3
Monkey See at time
13:30-14:15
Real Clients
Google
Server Kernel (cwin=3)
Web daemon
Tw1
Monkey See at time
14:16-15:06
Real Clients
Google
Server Kernel (cwin=1)
Web daemon
Treplay
Monkey Client/ Network Emulator
Monkey Do
Google
Server Kernel (cwin=3)
Monkey
Server Emulator
11/16/2018
USENIX '04

14. Predictive Replay Result
% connections
Search response time [ms]
CDF of search response times for Tw3, Tw1, and Treplay
11/16/2018
USENIX '04

15. Discussion Monkey can replay Google search traces well, but …
Can Monkey replay non-Google web or any TCP traffic?
Possible, if the assumptions hold
Caveats: dependencies in disk/network IO events, multiple TCP connections
Is server emulator required?
No, if the server performance distribution is “memory-less”. Otherwise, yes.
11/16/2018
USENIX '04

16. Conclusion
Monkey is a new tool for server benchmarking (“what if” questions)
Currently is domain-specific
Accurate for short flows
Best of both worlds
Realism of real user testing
Risk-free and convenience of synthetic tools
Possible to build such tools for specific domain
11/16/2018
USENIX '04

17. Q & A Questions? http://ramp.ucsd.edu/projects/monkey/ 11/16/2018
USENIX '04