Co* Projects : CoDNS, CoDeploy, CoMon KyoungSoo Park, Vivek Pai and Larry Peterson Princeton University 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Background : CoDeeN Academic Content Distribution Network 100+ proxy servers on PlanetLab Improve web performance & reliability Running 24/7 since June 2003 Roughly 3-4 million reqs/day aggregate Highest-traffic project on PlanetLab 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop How CoDeeN works Request Cache miss Response Cache Miss Response Cache hit Cache miss CoDeeN Proxy Response Cache hit Request Each CoDeeN proxy is a forward proxy, reverse proxy, & redirector 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

CoDeeN as Project Factory Deployment Issues Reliability/Security CoDNS Reliable DNS service CoDeploy Large file distribution Dynamic URL rewriting CoMon Monitoring infrastructure 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Local DNS Lookup Problems CoDeeN experiences DNS problems Local DNS lookup failures 5+ seconds delay for cached records Frequent & widely-distributed Local DNS lookups important LDNS cache hit rate: 80-90% Unpredictable service Directly affecting user-perceived latency Random delay in web browsing Critical in HTTP proxy, web crawlers and busy mail servers 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Experiment For Local Problems Local name lookup every 6 seconds “yyy.domain” on xxx.domain at Planetlab “planetlab-2.cs.princeton.edu” for planetlab-1.cs.princeton.edu Lookup should be handled locally Failure criteria 5+ seconds of latency zero answer Rolling average of the past 100 queries 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Expected DNS Behavior planetlab3.flux.utah.edu ricepl-1.cs.rice.pl 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

DNS Failure on Various Nodes planetlab1.cs.cornell.edu planetlab2.tamu.edu planetlab2.cs.uoregon.edu 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Possible Causes Packet loss LDNS overloading Cron jobs Maintenance problems 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Solution? CoDNS LDNS CoDNS remote queries remote answer LDNS CoDNS queries CoDNS LDNS x answer local node LDNS CoDNS 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop CoDNS Cooperative name lookup scheme If local server OK, use local server When failures, ask a peer for the lookup Selecting nearby peers Liveness/Remote nameserver’s health Send to improve cache locality Remote request timeout Dynamically adjusted to local server’s health Exponentially backed off for each remote query 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Evaluation Live traffic for one week for CoDeeN (20k - 30k) 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Finer-grained View Live traffic for one day Effectively flattens the spikes 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Availability Add one ‘9’, from 99% to 99.9% 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop For More Details CoDNS: Improving DNS Performance and Reliability via Cooperative Lookups KyoungSoo Park, Vivek S. Pai, Larry Peterson and Zhe Wang [OSDI04] Non-I2 nodes CDN names lookup CoDNS vs. TCP-based DNS 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop CoDeploy How to utilize the CDN infrastructure? Large file(100’s MB) distribution No custom protocol/software Preserves HTTP semantics Dynamic URL rewriting Supports Akamai style CDN CoralCDN [NSDI04] 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Large file distribution BitTorrent Bullet [SOSP03] SplitStream [SOSP03] FastReplica [USITS03] Download managers(Windows update, Adobe download manger) 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Drawback Availability Only BitTorrent is deployed But doesn’t really cache – peers are active downloaders Complexity Custom protocols Origin site needs special server, format Client software also custom 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Our Approach CDN file0-1 CDN file1-2 Agent CDN CDN file file2-3 Server file3-4 file4-5 CDN CDN 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Low-Level HTTP stuff GET name/ranges Header: blah HTTP/1.0 206 Partial Range: start-end/length GET name Range: bytes ranges Header: blah HTTP/1.0 200 OK Content-length: piece length New-header: obj length egress ingress 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Dynamic URL Rewriting http://www.host.com/path Client Server file http://proxy:port/capability/www.host.com/path Server Client CDN file rewritten file 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Benefit Transparently distributing the load Avoiding slashdot effect No setup at the browser Cheap way of publishing popular contents Easily customizable Capability: access control 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop CoMon PlanetLab monitoring infrastructure Node level statistics Uptime, cpu/memory usage, DNS failure rate Slice level statistics # proc, network BW, cpu/memory usage New slicestat Provides slice level resource consumption statisics e.g. curl http://localhost:3100/slicestat 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop Direction Which node is usable? sshable, DNS lookup, cpu/memory, file descriptor availability Which slice is going astray? What are the typical experiments? Automatic detection of overloading 9/18/2018 Co* Project 1st Asian PlanetLab Workshop

Co* Project 1st Asian PlanetLab Workshop More info Contact us! princeton_codeen@slices.planet-lab.org CoDeeN [USENIX04] http://codeen.cs.princeton.edu/ CoDNS [OSDI04] http://codeen.cs.princeton.edu/codns/ CoDeploy http://codeen.cs.princeton.edu/codeploy/ CoMon http://codeen.cs.princeton.edu/comon/ 9/18/2018 Co* Project 1st Asian PlanetLab Workshop