1. Co* Projects : CoDNS, CoDeploy, CoMon
KyoungSoo Park, Vivek Pai and Larry Peterson
Princeton University
9/18/2018
Co* Project st Asian PlanetLab Workshop

2. Co* Project 1st Asian PlanetLab Workshop
Background : CoDeeN
Academic Content Distribution Network
100+ proxy servers on PlanetLab
Improve web performance & reliability
Running 24/7 since June 2003
Roughly 3-4 million reqs/day aggregate
Highest-traffic project on PlanetLab
9/18/2018
Co* Project st Asian PlanetLab Workshop

3. Co* Project 1st Asian PlanetLab Workshop
How CoDeeN works
Request
Cache miss
Response
Cache Miss
Response
Cache hit
Cache miss
CoDeeN Proxy
Response
Cache hit
Request
Each CoDeeN proxy is a forward proxy, reverse proxy, & redirector
9/18/2018
Co* Project st Asian PlanetLab Workshop

4. CoDeeN as Project Factory
Deployment Issues
Reliability/Security
CoDNS
Reliable DNS service
CoDeploy
Large file distribution
Dynamic URL rewriting
CoMon
Monitoring infrastructure
9/18/2018
Co* Project st Asian PlanetLab Workshop

5. Local DNS Lookup Problems
CoDeeN experiences DNS problems
Local DNS lookup failures
5+ seconds delay for cached records
Frequent & widely-distributed
Local DNS lookups important
LDNS cache hit rate: 80-90%
Unpredictable service
Directly affecting user-perceived latency
Random delay in web browsing
Critical in HTTP proxy, web crawlers and busy mail servers
9/18/2018
Co* Project st Asian PlanetLab Workshop

6. Experiment For Local Problems
Local name lookup every 6 seconds
“yyy.domain” on xxx.domain at Planetlab
“planetlab-2.cs.princeton.edu” for planetlab-1.cs.princeton.edu
Lookup should be handled locally
Failure criteria
5+ seconds of latency
zero answer
Rolling average of the past 100 queries
9/18/2018
Co* Project st Asian PlanetLab Workshop

7. Co* Project 1st Asian PlanetLab Workshop
Expected DNS Behavior
planetlab3.flux.utah.edu
ricepl-1.cs.rice.pl
9/18/2018
Co* Project st Asian PlanetLab Workshop

8. DNS Failure on Various Nodes
planetlab1.cs.cornell.edu
planetlab2.tamu.edu
planetlab2.cs.uoregon.edu
9/18/2018
Co* Project st Asian PlanetLab Workshop

9. Co* Project 1st Asian PlanetLab Workshop
Possible Causes
Packet loss
LDNS overloading
Cron jobs
Maintenance problems
9/18/2018
Co* Project st Asian PlanetLab Workshop

10. Co* Project 1st Asian PlanetLab Workshop
Solution? CoDNS
LDNS
CoDNS
remote queries
remote answer
LDNS
CoDNS
queries
CoDNS
LDNS x
answer
local
node
LDNS
CoDNS
9/18/2018
Co* Project st Asian PlanetLab Workshop

11. Co* Project 1st Asian PlanetLab Workshop
CoDNS
Cooperative name lookup scheme
If local server OK, use local server
When failures, ask a peer for the lookup
Selecting nearby peers
Liveness/Remote nameserver’s health
Send to improve cache locality
Remote request timeout
Dynamically adjusted to local server’s health
Exponentially backed off for each remote query
9/18/2018
Co* Project st Asian PlanetLab Workshop

12. Co* Project 1st Asian PlanetLab Workshop
Evaluation
Live traffic for one week for CoDeeN (20k - 30k)
9/18/2018
Co* Project st Asian PlanetLab Workshop

13. Co* Project 1st Asian PlanetLab Workshop
Finer-grained View
Live traffic for one day
Effectively flattens the spikes
9/18/2018
Co* Project st Asian PlanetLab Workshop

14. Co* Project 1st Asian PlanetLab Workshop
Availability
Add one ‘9’, from 99% to 99.9%
9/18/2018
Co* Project st Asian PlanetLab Workshop

15. Co* Project 1st Asian PlanetLab Workshop
For More Details
CoDNS: Improving DNS Performance and Reliability via Cooperative Lookups KyoungSoo Park, Vivek S. Pai, Larry Peterson and Zhe Wang [OSDI04]
Non-I2 nodes
CDN names lookup
CoDNS vs. TCP-based DNS
9/18/2018
Co* Project st Asian PlanetLab Workshop

16. Co* Project 1st Asian PlanetLab Workshop
CoDeploy
How to utilize the CDN infrastructure?
Large file(100’s MB) distribution
No custom protocol/software
Preserves HTTP semantics
Dynamic URL rewriting
Supports Akamai style CDN
CoralCDN [NSDI04]
9/18/2018
Co* Project st Asian PlanetLab Workshop

17. Large file distribution
BitTorrent
Bullet [SOSP03]
SplitStream [SOSP03]
FastReplica [USITS03]
Download managers(Windows update, Adobe download manger)
9/18/2018
Co* Project st Asian PlanetLab Workshop

18. Co* Project 1st Asian PlanetLab Workshop
Drawback
Availability
Only BitTorrent is deployed
But doesn’t really cache – peers are active downloaders
Complexity
Custom protocols
Origin site needs special server, format
Client software also custom
9/18/2018
Co* Project st Asian PlanetLab Workshop

19. Co* Project 1st Asian PlanetLab Workshop
Our Approach
CDN
file0-1
CDN
file1-2
Agent
CDN
CDN
file
file2-3
Server
file3-4
file4-5
CDN
CDN
9/18/2018
Co* Project st Asian PlanetLab Workshop

20. Co* Project 1st Asian PlanetLab Workshop
Low-Level HTTP stuff
GET name/ranges
Header: blah
HTTP/ Partial
Range: start-end/length
GET name
Range: bytes ranges
Header: blah
HTTP/ OK
Content-length: piece length
New-header: obj length
egress
ingress
9/18/2018
Co* Project st Asian PlanetLab Workshop

21. Co* Project 1st Asian PlanetLab Workshop
Dynamic URL Rewriting
Client
Server
file
Server
Client
CDN
file
rewritten file
9/18/2018
Co* Project st Asian PlanetLab Workshop

22. Co* Project 1st Asian PlanetLab Workshop
Benefit
Transparently distributing the load
Avoiding slashdot effect
No setup at the browser
Cheap way of publishing popular contents
Easily customizable
Capability: access control
9/18/2018
Co* Project st Asian PlanetLab Workshop

23. Co* Project 1st Asian PlanetLab Workshop
CoMon
PlanetLab monitoring infrastructure
Node level statistics
Uptime, cpu/memory usage, DNS failure rate
Slice level statistics
# proc, network BW, cpu/memory usage
New slicestat
Provides slice level resource consumption statisics
e.g. curl
9/18/2018
Co* Project st Asian PlanetLab Workshop

24. Co* Project 1st Asian PlanetLab Workshop
9/18/2018
Co* Project st Asian PlanetLab Workshop

25. Co* Project 1st Asian PlanetLab Workshop
9/18/2018
Co* Project st Asian PlanetLab Workshop

26. Co* Project 1st Asian PlanetLab Workshop
9/18/2018
Co* Project st Asian PlanetLab Workshop

27. Co* Project 1st Asian PlanetLab Workshop
9/18/2018
Co* Project st Asian PlanetLab Workshop

28. Co* Project 1st Asian PlanetLab Workshop
Direction
Which node is usable?
sshable, DNS lookup, cpu/memory, file descriptor availability
Which slice is going astray?
What are the typical experiments?
Automatic detection of overloading
9/18/2018
Co* Project st Asian PlanetLab Workshop

29. Co* Project 1st Asian PlanetLab Workshop
More info
Contact us!
CoDeeN [USENIX04]
CoDNS [OSDI04]
CoDeploy
CoMon
9/18/2018
Co* Project st Asian PlanetLab Workshop