CoDoNs A High-Performance Alternative for the Domain Name System Emin Gün Sirer Venugopalan Ramasubramanian Computer Science, Cornell University

introduction caching is widely-used to improve latency and to decrease overhead passive caching caches distributed throughout the network store objects that are encountered not well-suited for a large-class applications

problems with passive caching no performance guarantees heavy-tail effect large percentage of queries to unpopular objects ad-hoc heuristics for cache management introduces coherency problems difficult to locate all copies weak consistency model

domain name system (dns) part of critical Internet infrastructure resolves names to addresses “  also resolves other flat queries hostname to mail server mappings hostname to owner information ipaddress to hostname and others based on a static hierarchy of servers

dns operation root edumilgovcomorg cornellupenn cs math ecearts wwwsyslab namespace partitioned into hierarchy lookup occurs from the top down through delegations static, manually-crafted, fragile hierarchy lookups are expensive no bounds! cnn ns1.verisign.com a.root-servers.net cit.cornell.edu ns.cs.cornell.edu

dns band-aids root edumilgovcomnet cornellupenn cs math ececitcit wwwsyslab replicate nodes at the top of the hierarchy 13 root servers 13 com, mil servers 2 org servers use passive caching for dns records at every possible location associate a TTL with every dns record no invalidations

dns problems performance bottleneck cold misses lead to high latencies vulnerable to DDoS attacks November 2002 attack on root servers can target branches of the name tree high load on nodes close to the root tree not a scalable data structure manual maintenance expensive, error prone DNS hierarchy full of errors coherency guarantees at odds with performance low TTL enables dynamic updates reduces performance

a better solution? distributed hash tables (DHTs) can help self-organizing scalable worst-case performance bounds but average-case performance for scalable DHTs is too high

0122 prefix-matching DHTs object = hash(“ map all nodes into an identifier space map all objects into same space based on key log b N hops several RTTs on the Internet

DHT properties scalable DHTs not suited for DNS Morris, Balakrishnan et al. O(log N) overlay hops is too slow for critical, latency-sensitive applications need faster lookups proactive, model-driven caching can help

Outline Problem passive caching DNS: a motivating example DHTs Beehive approach analytical model distributed implementation evaluation CoDoNs overview & operation evaluation Summary

overview of beehive general replication framework suitable for structured DHTs decentralization, self-organization, resilience properties high performance: O(1) average lookup time scalable: minimize number of replicas and reduce storage, bandwidth, and network load adaptive: promptly respond to changes in popularity – flash crowds

key intuition tunable latency adjust extent of replication for each object fundamental space- time tradeoff

analytical model optimization problem minimize: total number of replicas, s.t., average lookup performance  C configurable target lookup performance continuous range, sub one-hop minimizing number of replicas decreases storage and bandwidth overhead

target domain zipf-like query distributions number of queries to r th popular object  1/r  fraction of queries for m most popular objects  (m 1-  - 1) / (M 1-  - 1) commonly encountered in practice dns, web  typically between 0.80 and 1.00 MIT DNS trace = 0.91 traceDecUPisaFuNetUCBQuestNLANR 

level of replication nodes share i prefix-digits with the object i hop lookup latency lower level means greater extent of replication replicated on N/b i nodes

optimization problem minimize (storage/bandwidth) x 0 + x 1 /b + x 2 /b 2 + … + x K-1 /b K-1 such that (average lookup time is C hops) K – (x 0 1-  + x 1 1-  + x 2 1-  + … + x K-1 1-  )  C and x 0  x 1  x 2  …  x K-1  1 b: base K: log b (N) x i : fraction of objects replicated at level i

optimal closed-form solution d j (K’ – C) 1 + d + … + d K’  [ ] x* i =, 0  i  K’ – 1 where d = b (1-  ) / , K’  i  K K’ is determined by setting x* K’-1  1  d K’-1 (K’ – C) / (1 + d + … + d K’-1 )  1 1

implications O(1) average-case lookup performance C hops for Zipf-like distributions bounded worst-case K’ forms upper bound optimal number of replicas for   1 no wasted bandwidth, storage or coordination

latency - overhead trade off

beehive: system overview estimation popularity of objects, zipf parameter local measurement, limited aggregation replication apply analytical model independently at each node push new replicas to nodes at most one hop away

beehive replication protocol * home node EL * EBIL 2 0 * ABCDEFGHI L 1 object 0121

replication protocol low-overhead mechanisms distributed, no consensus asynchronous performed in the background modest state per object id (128 bits) home node indicator (1 bit) version number (64 bits)

mutable objects leverage the underlying structure of DHT replication level indicates the locations of all the replicas proactive propagation to all nodes from the home node home node sends to one-hop neighbors with i matching prefix-digits level i nodes send to level i+1 nodes

implementation and evaluation implemented using Pastry as the underlying DHT evaluation using a real DNS workload MIT DNS trace 1024 nodes, objects compared with passive caching on pastry main properties evaluated lookup performance storage and bandwidth overhead adaptation to changes in query distribution

time (hours) latency (hops) Pastry PC-Pastry Beehive evaluation: lookup performance passive caching is not very effective because of heavy tail query distribution and mutable objects. beehive converges to the target of 1 hop

evaluation: overhead x 10 6 time (hours) object transfers (#) PC-Pastry Beehive Object Transfers Storage average number of replicas per node Pastry40 PC-Pastry420 Beehive380

evaluation: flash crowds Latency time (hours) latency (hops) Pastry PC-Pastry Beehive

evaluation: zipf parameter change

Outline Problem passive caching DNS: a motivating example DHTs Beehive approach analytical model distributed implementation evaluation CoDoNs overview & operation evaluation Summary

Cooperative Domain Name System (CoDoNS) replacement for legacy DNS peer-to-peer DNS based on Beehive incremental deployment path completely transparent to clients uses legacy DNS to populate resource records on demand can operate without legacy DNS deployed on planet-lab

CoDoNs operation home node initially populates CoDoNs with binding from legacy DNS CoDoNs will propagate the optimal number of replicas Upon DNS record expiration, the home node checks binding for change

CoDoNs lookup performance

CoDoNSLegacy DNS median6.56 ms38.8 ms 90 th percentile281 ms337 ms

CoDoNs load balancing CoDoNs automatically balances load

CoDoNs security not an issue in a single administration domain e.g. akamai, google, msn, etc. attacks targeted at the DHT Castro et al. ’02 work on secure DHTs attacks targeted at Beehive outlier elimination limited impact attacks targeted at CoDoNs DNSSEC signatures

dns trickery server-side computation akamai and similar companies reroute clients to nearest nodes through dns dynamically computed records with low ttl not compatible with caching CoDoNs supports this hack by not caching entries whose TTL is lower than a threshold but better to perform server selection on the client side or not at all… CoDoNs already provides high average case performance and load balancing

CoDoNs implications name delegations can be purchased and propagated independently of server setup no static tree naming hierarchy independent of physical server hierarchy no manual configuration misconfiguration and broken delegations unlikely no ttl push updates at any time no verisign

advantages of CoDoNS high performance low lookup latency median latency of 7 ms for codons (planet-lab), 39 ms for legacy DNS resilience against denial of service attacks load balancing around hotspots self configuring around host and network failures fast, coherent updates no TTLs, updates can be propagated at any time

related work - DHTs Prefix-based Plaxton, Pastry, Chord, Tapestry O(log N) lookup, O(log N) storage deBruijn graphs Koorde, [Wieder & Naor 03] O(log N/log log N) lookup, O(log N) storage Butterfly Viceroy: O(log N) lookup, O(1) storage Farsite O(d) lookup, O(dn 1/d ) storage Kelips Kelips: O(1) lookup, O(  N) storage Full knowledge [Gupta, Liskov, Rodrigues]: O(1) lookup, O(N) storage

related work - DNS traces and zipf distributions web caching and zipf-like distributions: evidence and implications  Breslau, Cao, Fan, Phillips, and Shenker [infocom’99] popularity of gnutella queries and their implications on scalability  Sripanidkulchai [2002] caching and replication replication strategies in unstructured p2p networks  Cohen and Shenker [sigcomm’02] cup: controlled update propagation in p2p networks  Roussopoulos and Baker [usenix’03] DNS development of DNS system  Mockapetris [sigcomm’88] DNS performance and effectiveness of caching  Jung, Sit, Balakrishnan, and Morris [sigmetrics’01] serving DNS using a peer-to-peer lookup service  Cox, Muthitacharoen, and Morris [iptps’02]

conclusions model-driven proactive caching O(1) lookup performance with optimal replica cost beehive: a general replication framework structured overlays with uniform fan-out high performance, resilience, improved availability well-suited for latency sensitive applications CoDoNs: a modern replacement for DNS

evaluation: zipf parameter change

evaluation: instantaneous bandwidth overhead Object Transfers per sec time (hours) object transfers per sec PC-Pastry Beehive

typical values of zipf parameter MIT DNS trace:  = 0.91 Web traces: traceDecUPisaFuNetUCBQuestNLANR 

comparative overview of structured DHTs DHTlookup performance CANO(dN 1/d ) Chord, Kademlia, Pastry, Tapestry, Viceroy O(logN) de Bruijn graphs (Koorde)O(logN/loglogN) Kelips, Salad, [Gupta, Liskov, Rodriguez] O(1)

Beehive DNS: Lookup Performance CoDoNSLegacy DNS median6.56 ms38.8 ms 90 th percentile281 ms337 ms

introduction distributed peer-peer overlay networks decentralized self-organized distributed hash tables (DHTs) store – lookup interface unstructured DHTs Freenet, Gnutella, Kazaa bad lookup performance: accuracy and latency

example b = 32 C = 1  = 0.9 N = 10,000 M = 1,000,000 x* 0 = = 1102 objects x* 1 = = objects x* 2 = 1 total storage = 3700 objects per node total storage for Kelips = M/  N = 10,000 objects per node

structured overlays distributed peer-to-peer overlays decentralized, self-organized, resilient structured DHTs object storage and retrieval bounded average, worst-case latency latency sensitive applications domain name service (DNS) and web access need sub one hop latency

analytical model configurable target lookup performance continuous range even better with proximity routing minimizing number of replicas provides storage as well as bandwidth efficiency k’ is a upper bound on lookup performance of successful query assumptions homogeneous object sizes infrequent updates

beehive replication protocol periodic packets to nodes in routing table asynchronous and independent exploit structure of underlying DHT replication packet sent by node A to each node B in level i of routing table node B pushes new replicas to A and tells A which replicas to remove fluctuations in estimated popularity aging to prevent sudden changes hysteresis to limit thrashing

evaluation: DNS application DNS survey queried unique domain names TTL distribution: 95% < 1 day rate of change of entries: 0.13% per day MIT DNS trace: 4 ~ 11 december million queries for 300,000 distinct names zipf parameter: 0.91 setup simulation mode on single node 1024 nodes, distinct objects 7 queries per sec from MIT trace 0.8% per day rate of change

introduction lookup latency and storage CANChordPastryKelips latencyO(dN 1/d )O(log 2 N)O(log b N)O(1) 1,000,000 nodes 39.8 hops d = hops4.98 hops b = 16 ~1 hop 86.4 ms/hop 3.4 sec1.7 sec0.43 sec sec storageO(1/N) O(1/  N)

improving lookup latency passive caching of lookup results  not effective for heavy-tail query distributions  no guaranteed performance  updates invalidate cached results O(1) lookup performance trade off storage and bandwidth for performance Kelips: O(  N) replicas per object [GLR2003]: complete routing table

levels of replication ***** ***** level 0 3**** 3**** level 1 37** *level 2 obj id

optimal storage: C = 1 hop

summary and useful properties constant average lookup latency the constant is configurable popular objects have shorter lookup times? upper bounded by K’ (2 for  = 0.8) optimal overhead the storage and bandwidth requirements can be estimated overhead decreases with increasing  high availability for popular objects mitigates flash crowd effect proactive replication supports mutable objects more benefits can be derived by using proximity optimizations

a peer-peer DNS why p2p? iterative queries name-server mis-configurations  lots of failures and increased traffic less availability  chain of NS records update problem (Akamai) why BeeHive? constant lookup time  upper bound given by K’ (~2 or 3 hops)  comparable or better performance better availability due to replication support for mutability

DNS over beehive: distributed cooperative active cache deploy incrementally non-uniform rate of change scale popularity metric proportionately lookup failure negative caching reverse iterative resolution  lookup x.y.com, then y.com, then com…  fetches NS records  locality  inverse queries

DNS over BeeHive: security DNSSEC public key cryptography, signature chains namespace management  big sizes of key and sig records cache chain of key records for authentication popularity(y.com) > popularity(x.y.com) avoid duplicate key records authenticated denial reverse iterative resolution

other potential applications translation services semantic free lookup P6P p2p file sharing  text based search  anti-social applications? web  widely varying object sizes  dynamic content

conclusions BeeHive: p2p system based on differential replication goals efficient: constant lookup time with minimal overhead robust: self-organization and resilience against the vagaries of the network secure: resilience against malicious elements CoDoNS: Cooperative DNS on BeeHive

notations b:base of the DHT system N:number of nodes (b K ) M:number of objects  :alpha of zipf-like query distribution x j :fraction of objects replicated at level j or lower x 2 = fraction of objects replicated at levels 0, 1, and 2 0  x 0  x 1  x 2  …  x K-1  x K = 1

storage and bandwidth per node storage required at level j = M(x j – x j-1 )/b j total per node storage = Mx 0 + M(x 1 – x 0 )/b + M(x 2 – x 1 )/b 2 + … + M(x K – x K-1 )/b K = M [(1 – 1/b)(x 0 + x 1 /b + x 2 /b 2 + … + x K-1 /b K-1 ) + 1/b K ] total bandwidth = M b K [(1 – 1/b)(x 0 + x 1 /b + x 2 /b 2 + … + x K-1 /b K-1 )]

lookup latency fraction of queries for Mx j objects  ((Mx j ) 1-  - 1) / (M 1-  - 1)  x j 1-  average lookup time at level j  j (x j 1-  - x j-1 1-  ) average lookup time  (x 1 1-  - x 0 1-  ) + 2(x 2 1-  - x 1 1-  ) + … + K(x K 1-  - x K-1 1-  )  K – (x 0 1-  + x 1 1-  + x 2 1-  + … + x K-1 1-  )

optimization problem minimize x 0 + x 1 /b + x 2 /b 2 + … + x K-1 /b K-1 such that (x 0 1-  + x 1 1-  + x 2 1-  + … + x K-1 1-  )  K – C and x 0  x 1  x 2  …  x K-1  1 and x K-1  1

solution minimize x 0 + x 1 /b + x 2 /b 2 + … + x K’-1 /b K ’ -1 such that (x 0 1-  + x 1 1-  + x 2 1-  + … + x K’-1 1-  )  K’ – C solution using lagrange multiplier technique d j (K’ – C) 1 + d + … + d K’  [ ] x* j = 0  j  K’ – 1 d = b (1-  ) /  x* j = 1 K’  j  K K’ is determined by setting x* K’-1  1  d K’-1 (K’ – C) / (1 + d + … + d K’-1 )  1

constant lookup time systems kelips and scuttlebutt trade off bandwidth and storage for performance one-hop lookup latency resilience against transient nodes O(√N) replication of all objects  expensive update propagation