Evaluation of the Proximity between Web Clients and their Local DNS Servers Z. Morley Mao UC Berkeley Chuck Cranor, Fred Douglis, Michael Rabinovich, Oliver Spatscheck, and Jia Wang AT&T Labs--Research

Motivation Content Distribution Networks (CDNs) Try to deliver content from servers close to users Current server selection mechanisms Uses Domain Name System (DNS) Assumes that clients are close to their local DNS servers – “orginator problem” Verify the assumption that clients are close to their local DNS servers

Measurement setup Three components 1x1 pixel embedded transparent GIF image A specialized authoritative DNS server Allows hostnames to be wild-carded An HTTP redirector Always responds with “302 Moved Temporarily” Redirect to a URL with client IP address embedded

Embedded image request sequence Client [ ] Redirector for xxx.rd.example.com Local DNS server Content server for the image Name server for *.cs.example.com 1. HTTP GET request for the image 2. HTTP redirect to IP cs.example.com 3. Request to resolve IP cs.example.com 4. Request to resolve IP cs.example.com 5. Reply: IP address of content server 6. Reply: content server IP address 7. HTTP GET request for the image 8. HTTP response

Measurement impact Image (43 Byte) embedded at the end of the page, requested last Keynote measurement LocationWithout imageWith imageIncreased overhead World wide % US % Average download latency (sec)

Measurement Data SiteParticipantImage hit count Duration 1att.com20,816,9272 months 2,3Personal pages (commercial domain)1,7433 months 4AT&T research212,8143 months 5-7University sites4,367,0763 months 8-19Personal pages (university domain)26,5633 months

Measurement statistics Data typeCount Unique client-LDNS associations4,253,157 HTTP requests25,425,123 Unique client IPs3,234,449 Unique LDNS IPs157,633 Client-LDNS associations where Client and LDNS have the same IP address56,086

Top 10 busy ASes by request count AS numberOrganizationRequest count 7018AT&T876, UUNET779, AT&T BMGS239,989 1BBN Planet225, Sprint153, IBM145, Level 3143, Earthlink110, RoadRunner107,115

Proximity metrics: 1. AS, 2. network clustering AS clustering Observes if client and LDNS belong to the same AS Network clustering Network cluster based on BGP routing information using longest prefix match Observes if client and LDNS belong to the same network cluster

Proximity metric: 3. traceroute divergence Probe machine client Local DNS server Use the last point of divergence Traceroute divergence: Max(3,4)= a b

Proximity metric: 4. Roundtrip time correlation Correlation between message roundtrip times from a probe site to the client and its LDNS server The probe site represents a potential cache server location A crude metric, highly dependent on the probe site

Aggregate statistics of AS/network clustering About 12,000 Ases Observed close to 80% total ASes 440,000 unique prefixes 25% of all possible network clusters Metrics# client clusters # LDNS clusters Total # clusters AS clustering9,2158,5909,570 Network clustering98,00153,321104,950

Proximity analysis results: AS, network clustering MetricsClient IPsHTTP requests AS cluster64%69% Network cluster16%24% AS clustering: coarse-grained Network clustering: fine-grained Most clients not in the same routing entity as their LDNS Clients with LDNS in the same cluster slightly more active

Proximity analysis results: Traceroute divergence Probe sites: NJ(UUNET), NJ(AT&T), Berkeley(calren), Columbus(calren) Sampled from top half of busy network clusters Median divergence: 4 Mean divergence: Ratio of common to disjoint path length 72%-80% pairs traced have common path at least as long as disjoint path

Improved local DNS configuration For client-LDNS associations not in the same cluster, does there exist a LDNS in client’s cluster? MetricsOriginalImprovedOriginalImproved AS cluster64%88%69%92% Network cluster16%66%24%70% Client IPsHTTP requests

Clients using multiple LDNS A single client IP can be associated using multiple LDNS First LDNS times out, second contacted LDNS assigned dynamically through DHCP server LDNS configuration with multiple IPs Client IP reused by different users Client IP is the address of NAT or proxy Misconfiguration Majority of clients are associated with a single LDNS – 78%

Clients using 10 or fewer LDNS # clients (% total) # LDNS (avg # NAC) % total HTTP requests % associations in client’s NAC 2,524,939 (78.1)1 (1) ,228 (16.1)2 (1.6) ,524 (3.8)3 (2.1) ,422 (1.3)4 (2.5) ,469 (0.4)5 (2.9) ,555 (9.1)6 (3.3) ,590 (0.049)7 (4.1) (0.022)8 (4.7) (0.014)9 (5.5) (0.008)10 (6.1)

Client IPs using large number of LDNSs Common domain names: ( LDNS) *.MIL, apnc*, *bbnplanet.com, *hsacorp.net, *webcache.rcn.net, cache*.webcache.rcn.net, cache0*.proxy.aol.com, cache.brightok.net, cache*.ruh.isu.net.sa, *.onenet.net, hh*.direcpc.com, cob-cache.r.state.mn.us, mango.arctic.net, netcache.net.ca.gov, proxy.*.netsetter.com, *.nortelnetworks.com, rad.afonline.net, *.prserv.net, *.cisco.com, ss*.co.us.ibm.com, thing5.csc.com, *.wwwcache.ja.net

Example client IP using large number of LDNSs Client (proxy.sjc.netsetter.com) Using 241 LDNS 753 requests Belong to marketscore.com: Offers free browser plug-in for web acceleration Using client’s LDNS to do name resolution on behalf of client? HTTP headers: Via header: NetCache Network Appliance X-forwarded-for: , Client-ip: client IP address (dialup customers)

Top LDNS serving most clients DNS name# clients servedOrganization Ns?.worldnet.att.net68000AT&T Ns1.us.prserv.net42000IBM Nscache3.eng00.mindspring.net23000mindspring Rns2.earthlink.net17000Earthlink Lax1-dns.lax.netzero.net13000netzero Dns1.mtry01.pacbell.net12000Pac bell Ns.mia.bellsouth.net12000Bellsouth Dialcache040.ns.uu.net11000UUNET

Examination of clients from individual ASes Organization (AS #)AS clusterNetwork clusterNo. Reqs AT&T (7018)10%4%876,741 UUNET (6172)96%1%614,341 BBN (1)63%48%225,368 Sprint (1239)70%37%153,225 IBM (2688)3%0.5%145,158 UCB (25)98%34%38,196 MIT (3)99% 6,341 Cornell (26)99%46%2,341 CMU (9)99%94%4,090 UTAustin (18)98%70%12,878

Impact on commercial CDNs Impact on server selection accuracy Look for clients With LDNS responds to queries With a cache server in client’s cluster Whether directed to a cache server in a different cluster? – “misdirected”

Impact on commercial CDNs AS clustering CDNCDN XCDN YCDN Z Clients with CDN server in cluster 1,679,5151,215,372618,897 Verifiable clients1,324,022961,382516,969 Misdirected clients (% of verifiable clients) (% of clusters occupied) 809,683 (60%) (92%) 752,822 (77%) (94%) 434,905 (82%) (94%) Clients with LDNS not in client’s cluster (% of misdirected clients) 443,394 (55%) 354,928 (47%) 262,713 (60%)

Impact on commercial CDNs Network clustering CDNCDN XCDN YCDN Z Clients with cache server in cluster 264,743156,507103,448 Verifiable clients221,440132,56790,264 Misdirected clients (% of verifiable clients) (% of clusters occupied) 154,198 (68%) (77%) 125,449 (94%) (82%) 87,486 (96%) (93%) Clients with LDNS not in client’s cluster (% of misdirected clients) 145,276 (94%) 116,073 (93%) 84,737 (97%)

Why choosing a cache in a different cluster? Even when both client and LDNS are in the same cluster? Possible reasons Load-balancing algorithms using different metrics E.g., network access costs Caches are different Clustering too coarse-grained CDN mapping inaccuracies?

Lessons from study of commercial CDNs AS hop count is a bad metric for closeness evaluation too coarse-grained Maybe better choosing a geographically closer cache server in a different AS For load-balancing, fault-tolerance, CDNs sometimes return cache servers in two different Ases

Related work Measurement methodology 1. IBM (Shaikh et al.) Time correlation of DNS and HTTP requests from DNS and Web server logs 2. Univ of Boston (Bestavros et al.) Assigning multiple IP addresses to a Web server Differences from our work: Our methodology: efficient, accurate, nonintrusive 3. Web bugs Proximity metrics Cisco’s Boomerang protocol: uses latency from cache servers to the LDNS

Conclusion Novel technique for finding client and local DNS associations Fast, non-intrusive, and accurate DNS based server selection works well for coarse-grained load-balancing 64% associations in the same AS 16% associations in the same NAC Server selection can be inaccurate if server density is high