Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation 64-core CMP Motivation: Server Consolidation www server database server #1 database server #2 middleware server #1 Core L2 Cache L1

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation 64-core CMP Motivation: Server Consolidation www server database server #1 database server #2 middleware server #1

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation 64-core CMP Motivation: Server Consolidation www server database server #1 database server #2 middleware server #1 data Optimize Performance

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation 64-core CMP Motivation: Server Consolidation www server database server #1 database server #2 middleware server #1 Isolate Performance

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation 64-core CMP Motivation: Server Consolidation www server database server #1 database server #2 middleware server #1 Dynamic Partitioning

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation 64-core CMP Motivation: Server Consolidation www server database server #1 database server #2 middleware server #1 data Inter-VM Sharing VMWare’s Content-based Page Sharing  Up to 60% reduced memory

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Executive Summary Motivation: Server Consolidation Many-core CMPs increase opportunities Goals of Memory System: Performance Performance Isolation between VMs Dynamic Partitioning (VM Reassignment) Support Inter-VM Sharing Hypervisor/OS Simplicity Proposed Solution: Virtual Hierarchy Overlay 2-level hierarchy on physically-flat CMP Harmonize with VM Assignment

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Outline Motivation Server consolidation Memory system goals Non-hierarchical approaches Virtual Hierarchies Evaluation Conclusion

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation T AG -D IRECTORY duplicate tag directory A Read A

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation duplicate tag directory T AG -D IRECTORY A getM A 1 fwd data 3 duplicate tag directory 2 Read A A

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation S TATIC -B ANK -D IRECTORY getM A 1 2 fwd data 3 A A Read A

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation S TATIC -B ANK -D IRECTORY getM A 1 2 fwd data 3 A A Read A with hypervisor-managed cache

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Goals Optimize Performance Isolate Performance Allow Dynamic Partitioning Support Inter-VM Sharing Hypervisor/OS Simplicity Yes ? Yes No Yes S TATIC -B ANK -D IRECTORY T AG -D IRECTORY S TATIC -B ANK -D IRECTORY w/ hypervisor-managed cache No Yes

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Outline Motivation Virtual Hierarchies Evaluation Conclusion

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Virtual Hierarchies Key Idea: Overlay 2-level Coherence Hierarchy on CMP - First level harmonizes with VM/Workload - Second level allows inter-VM sharing, migration, reconfig

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation VH: First-Level Protocol Intra-VM Directory Protocol w/ interleaved directories Questions: How to name directories? How to name sharers? Dynamic home tile selected by VM Config Table Hardware VM Config Table at each tile Set by hypervisor during scheduling Full bit-vector to track any possible sharer Intra-VM broadcast also possible INV getM

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation VH: First-Level Protocol Example: Hypervisor/OS can freely change VM Config Table No cache flushes No atomic updates No explicit movement of directory state Address …… Home Tile: p14 offset 6 VM Config Table p12 p13 p p12 p13 p p13 p12 p14 Core L2 Cache L1 per-Tile Dynamic

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Virtual Hierarchies Two Solutions for Global Coherence: VH A and VH B memory controller(s)

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Protocol VH A Directory as Second-level Protocol Any tile can act as first-level directory How to track and name first-level directories? Full bit-vector of sharers to name any tile State stored in DRAM Possibly cache on-chip + Maximum flexibility - DRAM State - Complexity

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation VH A Example directory/memory controller getM A 1 2 data 6 A A Fwd data 4 3 getM A 5 Fwd data A

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Protocol VH B Broadcast as Second-level Protocol Attach token count for each block [token coherence] T tokens for each block. One token to read, all to write Allows 1-bit at memory per block Eliminates system-wide ACK + Minimal DRAM State + Enables easier optimizations - Global coherence requires more activity

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation VH B Example memory controller getM A global getM A getM A Data+tokens 4 A A

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Memory System Goals Optimize Performance Isolate Performance Allow Dynamic Partitioning Support Inter-VM Sharing Hypervisor/OS Simplicity VH A and VH B Yes

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Outline Motivation Virtual Hierarchies Evaluation Conclusion

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Evaluation: Methods Wisconsin GEMS Full-system, execution-driven simulation Based on Virtutech Simics 64-core tiled CMP In-order SPARC cores 512 KB, 16-way L2 cache per tile 2D mesh interconnect, 16-byte links, 5-cycle link latency Four on-chip memory controllers, 275-cycle DRAM latency

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Evaluation: Methods Workloads: OLTP, SpecJBB, Apache, Zeus Separate instance of Solaris for each VM Approximating Virtualization Multiple Simics checkpoints interleaved onto CMP Assume workloads map to adjacent cores Bottomline: No hypervisor simulated

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Evaluation: Protocols T AG -D IRECTORY : 3-cycle central tag directory (1024 ways!) S TATIC -B ANK -D IRECTORY Home tiles interleaved by frame address VH A All data allocates in L2 bank of dynamic home tile VH B Unshared data always allocates in local L2 All Protocols: one L2 copy of block on CMP

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Normalized Runtime Result: Runtime OLTP Apache T AG -D IR S TATIC -B ANK -D IR VH A VH B ZeusSpecJBB T AG -D IR S TATIC -B ANK -D IR VH A VH B T AG -D IR S TATIC -B ANK -D IR VH A VH B T AG -D IR S TATIC -B ANK -D IR VH A VH B Eight VMs x Eight Cores Each = 64 Cores (e.g. eight instances of Apache)

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Result: Memory Stall Cycles OLTPApache T AG -D IR S TATIC -B ANK -D IR VH A VH B Zeus SpecJBB T AG -D IR S TATIC -B ANK -D IR VH A VH B T AG -D IR S TATIC -B ANK -D IR VH A VH B T AG -D IR S TATIC -B ANK -D IR VH A VH B Eight VMs x Eight Cores Each = 64 cores (e.g. eight instances of Apache)

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Executive Summary Server Consolidation an Emerging Workload Goals of Memory System: Performance Performance Isolation between VMs Dynamic Partitioning (VM Reassignment) Support Inter-VM Sharing Hypervisor/OS Simplicity Proposed Solution: Virtual Hierarchy Overlay 2-level hierarchy on physically-flat CMP Harmonize with Workload Assignment

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Backup Slides

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Omitting 2 nd -Level Coherence: Protocol VH 0 Impacts: Dynamic Partitioning Inter-VM Sharing (VMWare’s Content-based Page Sharing) Hypervisor/OS complexity Example: Steps for VM Migration from Tiles {M} to {N} 1.Stop all threads on {M} 2.Flush {M} caches 3.Update {N} VM Config Tables 4.Start threads on {N}

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Omitting 2 nd -Level Coherence: Protocol VH 0 Example: Inter-VM Content-based Page Sharing Up to 60% reduced memory demand Is read-only sharing possible with VH 0 ? VMWare’s Implementation: Global hash table to store hashes of pages Guest pages scanned by VMM, hashes computed Full comparison of pages on hash match Potential VH 0 Implementation: How does hypervisor scan guest pages? Are they modified in cache? Even read-only pages must initially be written at some point

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Physical Hierarchy / Clusters P L1 $ P P P P P P P Shared L2 P L1 $ Shared L2 P L1 $ P P P P P P

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Physical Hierarchy / Clusters P L1 $ P P P P P P P Shared L2 P L1 $ Shared L2 P L1 $ P P P P P P www server database server #1 middleware server #1