1. The Google Cluster Architecture Written By: Luiz André Barroso Jeffrey Dean Urs Hölzle Presented By: Omkar Kasinadhuni Simerjeet Kaur

2. April 4th, 2007Google Cluster Architecture2 Agenda ► Introduction ► Architecture Overview  Serving a Google query  Using replication ► Leveraging Commodity Parts ► Power Problem ► Hardware-level applications characteristics  Memory system  Large-scale multiprocessing ► Conclusion

3. April 4th, 2007Google Cluster Architecture3 Introduction ► To build an infrastructure capable of executing thousands of queries per second, where each query in itself consumes billions of CPU cycles, is really challenging ► A single query on Google reads hundreds of megabytes of data and consumes tens of billions of CPU cycles. ► Google architecture features clusters of more than 15,000 commodity- class PCs with fault-tolerant software to achieve superior performance. ► Combining 15000 commodity-class PCs, with fault-tolerant software, gives higher performance than a smaller group of high-end servers.

4. April 4th, 2007Google Cluster Architecture4 ► Google benefits from on-chip parallelism (multithreading, on-chip multiprocessor) Important Design Factors: ► Energy Efficiency (considering thousands of machines) ► Price-Performance Ratio (not peak processor performance)

5. April 4th, 2007Google Cluster Architecture5 Google Architecture Overview ► Reliability in software (not in server-class hardware)  By replicating services ► Aggregate request throughput (not peak server response time)  Manage response time by parallelizing requests

6. April 4th, 2007Google Cluster Architecture6 Google search results for the query “ieee society”

7. April 4th, 2007Google Cluster Architecture7 Serving A Google Query ► Users enters query to Google ► User’s browser performs DNS lookup ► DNS based load balancing selects a cluster ► User’s browser sends HTTP request to the cluster ► Hardware based local load balancer balances the load among the Google Web Servers (GWSs) in that cluster. ► GWS machine sends the result to the user’s browser in HTML format.

8. April 4th, 2007Google Cluster Architecture8 Addresses: 209.85.165.104, 209.85.165.99, 209.85.165.147, 209.85.165.103 Aliases: www.google.com Grid Clusters Worldwide Clusters HTTP Request HTML Response DNS-based load-balancing DNS Lookup http://www.google.com/search?q=ieee+societyhttp://www.google.com/search?q=ieee+society -> IP Cluster addresses returned

9. April 4th, 2007Google Cluster Architecture9 Serving A Google Query Internally GWS machine coordinates query execution in two phases: Phase 1: (Index lookup phase) ► Index Servers use inverted index  Maps each query word to matching documents (hit list) ► Compute relevance score for documents  Score determines the order of results on the output page. ► Search process is challenging  Tens of terabytes of uncompressed data in documents  Inverted index itself results in many terabytes of data

10. April 4th, 2007Google Cluster Architecture10 Serving A Google Query Search is highly Parallelizable: ► Divide Index into pieces – index shards ► Each shard is served by pool of machines ► An intermediate load balancer chooses machine (or subset of machines) ► Even if a machine fails, the service remains uninterrupted ► The final result of the first phase is an ordered list of document identifiers (docids)

11. April 4th, 2007Google Cluster Architecture11 Serving A Google Query Index (a-z) shard-1 (a - h) shard-2 (i - o) Shard - (n-1) (q - v) Shard – n (w - z) ……. Figure: Index Shards ………

12. April 4th, 2007Google Cluster Architecture12 Google Web Servers (GWSs) Figure 1: Google query-serving architecture

13. April 4th, 2007Google Cluster Architecture13 Google Web Servers (GWSs) Phase 2: (Document serving phase) ► Use the list of docids to generate actual title, URL, query specific document summary ► Document servers (docservers) fetch each document from the disk and extract title and keyword-in-context snippet. ► The docservers processes the documents by partitioning them just as in the Index lookup phase. Query specific document summary Title URL

14. April 4th, 2007Google Cluster Architecture14 Google Web Servers (GWSs) ► The docserver cluster must have access to an online copy of the entire Web. ► The GWS also initiates tasks like spell-checker, ad-server. Figure 1: Google query-serving architecture

15. April 4th, 2007Google Cluster Architecture15 Using Replication ► Google maintains several copies of the entire Web across its clusters for performance and availability issues ► Most of the accesses to the indexes are read-only. Updates are possible by diverting queries away from service replica ► Adding machines to each pool increases serving capacity and adding shards accommodates index growth

16. April 4th, 2007Google Cluster Architecture16 ► Average latency is reduced by parallelizing search ► Individual shards do not require communication resulting speedup to be nearly linear. ► CPU speed of the individual index servers does not directly influence the search’s overall performance Using Replication

17. April 4th, 2007Google Cluster Architecture17 ► Software reliability ► Use replication for better request throughput and availability ► Price/Performance beats peak performance ► Using commodity PCs reduces the cost of computation Google Cluster Design Principles

18. April 4th, 2007Google Cluster Architecture18 Leveraging Commodity Parts ► Google's racks consists of 40 to 80 x86-based servers on custom made racks ► Several CPUs generations are used from single-processor 533-Mhz Intel-Celeron to dual 1.4-Ghz Intel Pentium III ► Servers contains one or more 80GB IDE drives ► Index Servers have less disk space than document servers ► Servers on rack are interconnected via 100-Mbps Ethernet switch ► A core gigabit switch connects all racks together

19. April 4th, 2007Google Cluster Architecture19 ► The selection criteria is cost per query, i.e. sum of capital expenses (with depreciation) and operating costs (hosting, administration, repairs) divided by performance  1 Rack ($287,000) ► Capital cost $278,000 for 3 years ► Operating cost $9,000 for 3 years  Queries ► 1 second = 1000 ► 3 years = 3.942 * 10 9  Cost per query = 287000 / 3.942 * 10 9 = 0.000082 cents ► The load balancing between current generation (faster) machines and older machines is difficult ► Relatively short amortization period causes the equipment cost have a significant impact on overall cost equation Leveraging Commodity Parts

20. April 4th, 2007Google Cluster Architecture20 Leveraging Commodity Parts ► Rack configuration : (costs about $278,000)  88 dual-CPU, 2-GHz Intel Xeon servers,  2 GB RAM and  80 GB hard disk ► Monthly capital cost of $7,700 over 3 years ► In total, 176 2-GHz Xeon CPUs, 176 GB of RAM, and 7 TB of disk space ► A typical x86-based system: (costs about $758,000)  Eight 2-GHz Xeon CPUs,  64 GB of RAM and  8 TB of hard disk space. ► In comparison, multiprocessor server is about 3 times more expensive but has 22 times fewer CPUs, 3 times less RAM and slightly more disk space.

21. April 4th, 2007Google Cluster Architecture21 Leveraging Commodity Parts ► Expensive equipment increase performance but decrease price per performance ► High-end servers provide high interconnect bandwidth and reliability which Goggle achieves with its highly redundant architecture ► Managing thousands of PCs instead of few servers incurs system administration and repair costs, but Goggle's homogeneous application makes these costs manageable

22. April 4th, 2007Google Cluster Architecture22 The Power Problem ► A mid-range server  requires 90 W of DC power ► 55 W - 2 CPUs, ► 10 W - disk drive, ► 25 W-DRAM and motherboard  120 W of AC power ► 10 kW per rack ► Power density = 400 W/ft 2 (1 rack-25 ft 2 ) ► Power density (high-end processors) = 700 W/ft 2 ► A commercial data center typically holds power densities between 70 to 150 W/ft 2

23. April 4th, 2007Google Cluster Architecture23 The Power Problem ► Low-power servers must not be very expensive ► Watts per unit of performance is important than watt alone ► 10 kW rack consumes 10 MW-h of power per month (including cooling overhead) ► With 15 cents per kW-hour, power and cooling costs are $1,500 per month (much less than $7,700)

24. April 4th, 2007Google Cluster Architecture24 Hardware-level Applications Characteristics ► Index servers contribute the most in the overall price performance ► There isn’t much exploitable instruction level parallelism in the workload ► Profitable way to exploit parallelism in index server is to benefit from easily parallelizable computation ► Both simultaneous multithreading (SMT) and chip multiprocessor (CMP) target thread level parallelism

25. April 4th, 2007Google Cluster Architecture25 Hardware-level Applications Characteristics ► CMP yields more performance than SMT ► CMP systems use short pipeline cores that reduces branch mispredict penalties. ► Thread level parallelism should allow linear speedup with number of cores ► L2 cache speeds up interprocessor communication

26. April 4th, 2007Google Cluster Architecture26 Memory System ► Index data block have no temporal locality ► Access within an index data block benefit from spatial locality ► Even for modest cache sizes, good overall cache hit ratios is achieved ► Memory bandwidth is not a bottleneck ► The memory utilization of Pentium- class processor system is under 20 percent ► Modest sized L2 cache or short L2 cache and memory latencies, and longer cache lines are most efficient for Google workload

27. April 4th, 2007Google Cluster Architecture27 Large Scale Multiprocessing Large-scale shared memory machines are useful when: ► Computation to communication ratio is low ► Communication patterns or data partitioning are dynamic or hard to predict ► Cost of ownership gets higher than hardware costs The above requirements do not apply to Google

28. April 4th, 2007Google Cluster Architecture28 ► All the software is produced in-house ► System management overhead is minimized through automation and monitoring ► Google’s hardware cost is only a small fraction of the total system operating expenses Large Scale Multiprocessing

29. April 4th, 2007Google Cluster Architecture29 Conclusion ► The cluster solution is best suited for performance and availability. ► The system achieves high fault tolerance due to redundancy (not because of its hardware capabilities) ► An application which focuses on the price/performance and can run on servers that have no private state might benefit from similar architecture.

30. April 4th, 2007Google Cluster Architecture30 Discussion ► The disk capacity in each cluster grows at some rate. The web grows at some other rate. Is the rate at which the Web grows a potential problem? ► Cost per query is an important factor in Google Architecture, how would you relate it with cost per advertisement? ► At Google’s scale of operation, what are the complexities of data storage management?