The Google Cluster Architecture Written By: Luiz André Barroso Jeffrey Dean Urs Hölzle Presented By: Omkar Kasinadhuni Simerjeet Kaur

April 4th, 2007Google Cluster Architecture2 Agenda ► Introduction ► Architecture Overview  Serving a Google query  Using replication ► Leveraging Commodity Parts ► Power Problem ► Hardware-level applications characteristics  Memory system  Large-scale multiprocessing ► Conclusion

April 4th, 2007Google Cluster Architecture3 Introduction ► To build an infrastructure capable of executing thousands of queries per second, where each query in itself consumes billions of CPU cycles, is really challenging ► A single query on Google reads hundreds of megabytes of data and consumes tens of billions of CPU cycles. ► Google architecture features clusters of more than 15,000 commodity- class PCs with fault-tolerant software to achieve superior performance. ► Combining commodity-class PCs, with fault-tolerant software, gives higher performance than a smaller group of high-end servers.

April 4th, 2007Google Cluster Architecture4 ► Google benefits from on-chip parallelism (multithreading, on-chip multiprocessor) Important Design Factors: ► Energy Efficiency (considering thousands of machines) ► Price-Performance Ratio (not peak processor performance)

April 4th, 2007Google Cluster Architecture5 Google Architecture Overview ► Reliability in software (not in server-class hardware)  By replicating services ► Aggregate request throughput (not peak server response time)  Manage response time by parallelizing requests

April 4th, 2007Google Cluster Architecture6 Google search results for the query “ieee society”

April 4th, 2007Google Cluster Architecture7 Serving A Google Query ► Users enters query to Google ► User’s browser performs DNS lookup ► DNS based load balancing selects a cluster ► User’s browser sends HTTP request to the cluster ► Hardware based local load balancer balances the load among the Google Web Servers (GWSs) in that cluster. ► GWS machine sends the result to the user’s browser in HTML format.

April 4th, 2007Google Cluster Architecture8 Addresses: , , , Aliases: Grid Clusters Worldwide Clusters HTTP Request HTML Response DNS-based load-balancing DNS Lookup -> IP Cluster addresses returned

April 4th, 2007Google Cluster Architecture9 Serving A Google Query Internally GWS machine coordinates query execution in two phases: Phase 1: (Index lookup phase) ► Index Servers use inverted index  Maps each query word to matching documents (hit list) ► Compute relevance score for documents  Score determines the order of results on the output page. ► Search process is challenging  Tens of terabytes of uncompressed data in documents  Inverted index itself results in many terabytes of data

April 4th, 2007Google Cluster Architecture10 Serving A Google Query Search is highly Parallelizable: ► Divide Index into pieces – index shards ► Each shard is served by pool of machines ► An intermediate load balancer chooses machine (or subset of machines) ► Even if a machine fails, the service remains uninterrupted ► The final result of the first phase is an ordered list of document identifiers (docids)

April 4th, 2007Google Cluster Architecture11 Serving A Google Query Index (a-z) shard-1 (a - h) shard-2 (i - o) Shard - (n-1) (q - v) Shard – n (w - z) ……. Figure: Index Shards ………

April 4th, 2007Google Cluster Architecture12 Google Web Servers (GWSs) Figure 1: Google query-serving architecture

April 4th, 2007Google Cluster Architecture13 Google Web Servers (GWSs) Phase 2: (Document serving phase) ► Use the list of docids to generate actual title, URL, query specific document summary ► Document servers (docservers) fetch each document from the disk and extract title and keyword-in-context snippet. ► The docservers processes the documents by partitioning them just as in the Index lookup phase. Query specific document summary Title URL

April 4th, 2007Google Cluster Architecture14 Google Web Servers (GWSs) ► The docserver cluster must have access to an online copy of the entire Web. ► The GWS also initiates tasks like spell-checker, ad-server. Figure 1: Google query-serving architecture

April 4th, 2007Google Cluster Architecture15 Using Replication ► Google maintains several copies of the entire Web across its clusters for performance and availability issues ► Most of the accesses to the indexes are read-only. Updates are possible by diverting queries away from service replica ► Adding machines to each pool increases serving capacity and adding shards accommodates index growth

April 4th, 2007Google Cluster Architecture16 ► Average latency is reduced by parallelizing search ► Individual shards do not require communication resulting speedup to be nearly linear. ► CPU speed of the individual index servers does not directly influence the search’s overall performance Using Replication

April 4th, 2007Google Cluster Architecture17 ► Software reliability ► Use replication for better request throughput and availability ► Price/Performance beats peak performance ► Using commodity PCs reduces the cost of computation Google Cluster Design Principles

April 4th, 2007Google Cluster Architecture18 Leveraging Commodity Parts ► Google's racks consists of 40 to 80 x86-based servers on custom made racks ► Several CPUs generations are used from single-processor 533-Mhz Intel-Celeron to dual 1.4-Ghz Intel Pentium III ► Servers contains one or more 80GB IDE drives ► Index Servers have less disk space than document servers ► Servers on rack are interconnected via 100-Mbps Ethernet switch ► A core gigabit switch connects all racks together

April 4th, 2007Google Cluster Architecture19 ► The selection criteria is cost per query, i.e. sum of capital expenses (with depreciation) and operating costs (hosting, administration, repairs) divided by performance  1 Rack ($287,000) ► Capital cost $278,000 for 3 years ► Operating cost $9,000 for 3 years  Queries ► 1 second = 1000 ► 3 years = * 10 9  Cost per query = / * 10 9 = cents ► The load balancing between current generation (faster) machines and older machines is difficult ► Relatively short amortization period causes the equipment cost have a significant impact on overall cost equation Leveraging Commodity Parts

April 4th, 2007Google Cluster Architecture20 Leveraging Commodity Parts ► Rack configuration : (costs about $278,000)  88 dual-CPU, 2-GHz Intel Xeon servers,  2 GB RAM and  80 GB hard disk ► Monthly capital cost of $7,700 over 3 years ► In total, GHz Xeon CPUs, 176 GB of RAM, and 7 TB of disk space ► A typical x86-based system: (costs about $758,000)  Eight 2-GHz Xeon CPUs,  64 GB of RAM and  8 TB of hard disk space. ► In comparison, multiprocessor server is about 3 times more expensive but has 22 times fewer CPUs, 3 times less RAM and slightly more disk space.

April 4th, 2007Google Cluster Architecture21 Leveraging Commodity Parts ► Expensive equipment increase performance but decrease price per performance ► High-end servers provide high interconnect bandwidth and reliability which Goggle achieves with its highly redundant architecture ► Managing thousands of PCs instead of few servers incurs system administration and repair costs, but Goggle's homogeneous application makes these costs manageable

April 4th, 2007Google Cluster Architecture22 The Power Problem ► A mid-range server  requires 90 W of DC power ► 55 W - 2 CPUs, ► 10 W - disk drive, ► 25 W-DRAM and motherboard  120 W of AC power ► 10 kW per rack ► Power density = 400 W/ft 2 (1 rack-25 ft 2 ) ► Power density (high-end processors) = 700 W/ft 2 ► A commercial data center typically holds power densities between 70 to 150 W/ft 2

April 4th, 2007Google Cluster Architecture23 The Power Problem ► Low-power servers must not be very expensive ► Watts per unit of performance is important than watt alone ► 10 kW rack consumes 10 MW-h of power per month (including cooling overhead) ► With 15 cents per kW-hour, power and cooling costs are $1,500 per month (much less than $7,700)

April 4th, 2007Google Cluster Architecture24 Hardware-level Applications Characteristics ► Index servers contribute the most in the overall price performance ► There isn’t much exploitable instruction level parallelism in the workload ► Profitable way to exploit parallelism in index server is to benefit from easily parallelizable computation ► Both simultaneous multithreading (SMT) and chip multiprocessor (CMP) target thread level parallelism

April 4th, 2007Google Cluster Architecture25 Hardware-level Applications Characteristics ► CMP yields more performance than SMT ► CMP systems use short pipeline cores that reduces branch mispredict penalties. ► Thread level parallelism should allow linear speedup with number of cores ► L2 cache speeds up interprocessor communication

April 4th, 2007Google Cluster Architecture26 Memory System ► Index data block have no temporal locality ► Access within an index data block benefit from spatial locality ► Even for modest cache sizes, good overall cache hit ratios is achieved ► Memory bandwidth is not a bottleneck ► The memory utilization of Pentium- class processor system is under 20 percent ► Modest sized L2 cache or short L2 cache and memory latencies, and longer cache lines are most efficient for Google workload

April 4th, 2007Google Cluster Architecture27 Large Scale Multiprocessing Large-scale shared memory machines are useful when: ► Computation to communication ratio is low ► Communication patterns or data partitioning are dynamic or hard to predict ► Cost of ownership gets higher than hardware costs The above requirements do not apply to Google

April 4th, 2007Google Cluster Architecture28 ► All the software is produced in-house ► System management overhead is minimized through automation and monitoring ► Google’s hardware cost is only a small fraction of the total system operating expenses Large Scale Multiprocessing

April 4th, 2007Google Cluster Architecture29 Conclusion ► The cluster solution is best suited for performance and availability. ► The system achieves high fault tolerance due to redundancy (not because of its hardware capabilities) ► An application which focuses on the price/performance and can run on servers that have no private state might benefit from similar architecture.

April 4th, 2007Google Cluster Architecture30 Discussion ► The disk capacity in each cluster grows at some rate. The web grows at some other rate. Is the rate at which the Web grows a potential problem? ► Cost per query is an important factor in Google Architecture, how would you relate it with cost per advertisement? ► At Google’s scale of operation, what are the complexities of data storage management?