Download presentation
Presentation is loading. Please wait.
1
How Akamai Handles Large Events Bruce Maggs Carnegie Mellon Duke Akamai Technologies
2
Traffic Statistics 30,000+ domains 1.1 Tbps daily peak traffic 6,419 terabytes / day 274 billion hits / day 274 million unique client IP addresses / day in 2009 expect to deliver more bits than in 1998-2008 combined
3
Network Deployment 40000+ Servers 950+ Networks 67+ Countries 1450+ POPs
4
Assigning Clients to Servers “High level”: Map client to a cluster based on client’s nameserver’s IP address. Algorithm: stable marriage with multi-dimensional hierarchical capacity constraints.“High level”: Map client to a cluster based on client’s nameserver’s IP address. Algorithm: stable marriage with multi-dimensional hierarchical capacity constraints. “Low level”: Assign client to specific server or servers within cluster based on content requested. Algorithm: consistent hashing.“Low level”: Assign client to specific server or servers within cluster based on content requested. Algorithm: consistent hashing.
5
Embedded Image Delivery (e.g., Amazon) <html><head> Welcome to xyz.com! Welcome to xyz.com! </head><body> <img src=“ Welcome to our Web site! Welcome to our Web site! Click here to enter Click here to enter </body></html> http://www.xyz.com/logos/logo.gif”> http://www.xyz.com/jpgs/navbar1.jpg”> Embedded URLs are Converted to ARLs ak
6
End User Akamai DNS Resolution Akamai High-Level DNS Servers 4 ak.xyz.com 1 Browser’s Cache OS 2 Local Name Server 3 10 9 5 20.20.123.55 Akamai Low-Level DNS Servers 6 ak.xyz.com 30.30.123.5 7 8 select cluster select servers within cluster
7
Mapping Maps IP address of client’s name server and type of content being requested to an Akamai cluster.Maps IP address of client’s name server and type of content being requested to an Akamai cluster. Note: Doesn’t depend on content provider (although indicated by ak.xyz.com).Note: Doesn’t depend on content provider (although indicated by ak.xyz.com). Special cases: Akamai Accelerated Network Partners (AANPs)Special cases: Akamai Accelerated Network Partners (AANPs) General case: “Core Point” analysisGeneral case: “Core Point” analysis
8
Core Points Core point X is the first router at which all paths to nameservers 1, 2, 3, and 4 intersect.Core point X is the first router at which all paths to nameservers 1, 2, 3, and 4 intersect. Traceroute once per day from 300 clusters to 280,000 nameservers.Traceroute once per day from 300 clusters to 280,000 nameservers. X 1 23 4 Akamai cluster 3 Akamai cluster 2 Akamai cluster 1
9
Core Points 280,000 nameservers (98.8% of requests) reduced to 30,000 core points 30,000 core points ping core points every 6 minutes
10
Akamai Cluster Servers pool resources RAM Disk Throughput
11
Hashing E.g., h(x) = (((a x + b) mod P) mod |B|), where P is prime, P > |U| a,b chosen uniformly at random from Z P x is a serial number (designates content provider) Universe U of all possible objects, set B of buckets. object: set of web objects with same serial number bucket: web server Hash function h: U B Assigns objects to buckets
12
f(d) = d + 1 mod 5 Difficulty changing number of buckets 5 7 10 11 27 29 36 38 40 43 4321043210 bucket object f(d) = d + 1 mod 4
13
Consistent Hashing Idea: Map both objects and buckets to unit circle. object bucket Assign object to next bucket on circle in clockwise order. new bucket
14
How it’s really done a212: 10.10.10.1 10.10.10.4 10.10.10.3 10.10.10.2 a213: 10.10.10.3 10.10.10.4 10.10.10.2 10.10.10.1 a214: 10.10.10.1 10.10.10.2 10.10.10.3 10.10.10.4 a215: 10.10.10.2 10.10.10.1 10.10.10.4 10.10.10.3 random permutations of servers Why? To spread load for one serial number.
15
Strategies for Handling Large Events Flash crowds (no advance notice): Rely on rapid load balancing. Cross fingers.Flash crowds (no advance notice): Rely on rapid load balancing. Cross fingers. Planned events: “pre-warm” caches with objects to be served.Planned events: “pre-warm” caches with objects to be served.
16
Content Delivery on 9/11 (2001) Akamai’s network had capacity for all content providers requesting serviceAkamai’s network had capacity for all content providers requesting service Total bits served on September 11 was approximately 3.5 times normalTotal bits served on September 11 was approximately 3.5 times normal Traffic was higher on September 12Traffic was higher on September 12 (But not as high as January 7, 2002)(But not as high as January 7, 2002)
17
News Site A – Image Traffic
18
News Site A – Streaming
19
News Site B – HTML Traffic
20
News Site B – Image Traffic
21
News Site B – Streaming
22
Portal A – Image Traffic
23
Sports Site A – Image Traffic
24
Steve Jobs Keynote (2002)
25
Akamai live streaming infrastructure AAAAAA Reflectors Edge servers Source Trace format [Client IP, Player ID, Stream URL, Session start time, Session duration, Mean receiving bandwidth]
![Akamai live streaming infrastructure AAAAAA Reflectors Edge servers Source Trace format [Client IP, Player ID, Stream URL, Session start time, Session duration, Mean receiving bandwidth]](http://images.slideplayer.com./25/7602535/slides/slide_25.jpg "Akamai live streaming infrastructure AAAAAA Reflectors Edge servers Source Trace format [Client IP, Player ID, Stream URL, Session start time, Session duration, Mean receiving bandwidth]")
26
Extensive traces (2004) ~1,000,000 daily requests ~200,000 daily client IP addresses from over 200 countries ~1,000 daily streams ~1,000 edge servers ~Everyday, over a 3-month period
27
Audio vs. video Unknown 22% Audio 71% Video 7% Most streams are audio.
28
Stream types Non-stop (76%) vs. short duration (24%) –All video streams have short duration Smooth arrivals (50%) vs. sudden onset (50%) –Flash crowds are common
29
Obama Inauguration (2009) 7 million simultaneous data streams, most carrying live video total traffic peaking at 2 terabits per second requests per minute fifth place historically, behind election night in November and three sports events (a World Cup soccer playoff and two NCAA basketball playoff games)
30
© 2008 Akamai PROPRIETARY 2: Network and Link Characterization Before inauguration
31
© 2008 Akamai PROPRIETARY 2: Network and Link Characterization Just after – the Internet’s performance starts to degrade massively
32
© 2008 Akamai PROPRIETARY 2: Network and Link Characterization And later is mostly better but not great
Similar presentations
