1 Computation Spreading: Employing Hardware Migration to Specialize CMP Cores On-the-fly Koushik Chakraborty Philip Wells Gurindar Sohi

Chakraborty, Wells, and Sohi ASPLOS Paper Overview  Multiprocessor Code Reuse Poor resource utilization  Computation Spreading New model for assigning computation within a program on CMP cores in H/W Case Study: OS and User computation  Investigate performance characteristics

Chakraborty, Wells, and Sohi ASPLOS Talk Outline  Motivation  Computation Spreading (CSP) Case study: OS and User compution  Implementation  Results  Related Work and Summary

Chakraborty, Wells, and Sohi ASPLOS Homogeneous CMP  Many existing systems are homogeneous Sun Niagara, IBM Power 5, Intel Xeon MP  Multithreaded server application Composed of server threads Typically each thread handles a client request OS assigns software threads to cores Entire computation from one thread execute on a single core (barring migration)

Chakraborty, Wells, and Sohi ASPLOS Code Reuse  Many client requests are similar Similar service across multiple threads Same code path traversed in multiple cores  Instruction footprint classification Exclusive – single core access Common – many cores access Universal – all cores access

Chakraborty, Wells, and Sohi ASPLOS Multiprocessor Code Reuse

Chakraborty, Wells, and Sohi ASPLOS Implications  Lack of instruction stream specialization Redundancy in predictive structures Poor capacity utilization Destructive interference  No synergy among multiple cores Lost opportunity for co-operation Exploit core proximity in CMP

Chakraborty, Wells, and Sohi ASPLOS Talk Outline  Motivation  Computation Spreading (CSP) Case study: OS and User compution  Implementation  Results  Related Work and Summary

Chakraborty, Wells, and Sohi ASPLOS Computation Spreading (CSP)  Computation fragment = dynamic instruction stream portion  Collocate similar computation fragments from multiple threads Enhance constructive interference  Distribute dissimilar computation fragments from a single thread Reduce destructive interference Reassignment is the key

Chakraborty, Wells, and Sohi ASPLOS Example A1B1C1A1B1C1 B2C2A2B2C2A2 C3A3B3C3A3B3 T1 T2 T3 B3B3 A3A3 C3C3 A1A1 C1C1 B1B1 B2B2 C2C2 A2A2 P1 P2 P3 CANONICAL CSP time A1B1C1A1B1C1 B2C2A2B2C2A2 C3A3B3C3A3B3

Chakraborty, Wells, and Sohi ASPLOS Key Aspects  Dynamic Specialization Homogeneous multicore acquires specialization via retaining mutually exclusive predictive state  Data Locality Data dependencies between different computation fragments Careful fragment selection to avoid loss of data locality

Chakraborty, Wells, and Sohi ASPLOS Selecting Fragments  Server workloads characteristics Large data and instruction footprint Significant OS computation  User Computation and OS Computation A natural separation Exclusive instruction footprints Relatively independent Relatively independent data footprint

Chakraborty, Wells, and Sohi ASPLOS Data Communication T1T1 T2T2 T 1 -User T 1 -OS T 2 -User T 2 -OS Core 1Core 2

Chakraborty, Wells, and Sohi ASPLOS Relative Inter-core Data Communication ApacheOLTP OS-User Communication is limited

Chakraborty, Wells, and Sohi ASPLOS Talk Outline  Motivation  Computation Spreading (CSP) Case study: OS and User compution  Implementation  Results  Related Work and Summary

Chakraborty, Wells, and Sohi ASPLOS Implementation  Migrating Computation Transfer state through the memory subsystem ~2KB of register state in SPARC V9 Memory state through coherence  Lightweight Virtual Machine Monitor Migrates computation as dictated by the CSP Policy Implemented in hardware/firmware

Chakraborty, Wells, and Sohi ASPLOS Baseline User Cores OS Cores User Comp OS Comp Virtual CPUs Physical Cores Software Stack Implementation cont Threads

Chakraborty, Wells, and Sohi ASPLOS User Cores OS Cores Virtual CPUs Physical Cores Software Stack Implementation cont Threads

Chakraborty, Wells, and Sohi ASPLOS CSP Policy  Policy dictates computation assignment  Thread Assignment Policy (TAP) Maintains affinity between VCPUs and physical cores  Syscall Assignment Policy (SAP) OS computation assigned based on system calls  TAP and SAP use identical assignment for user computation

Chakraborty, Wells, and Sohi ASPLOS Talk Outline  Motivation  Computation Spreading (CSP) Case study: OS and User compution  Implementation  Results  Related Work and Summary

Chakraborty, Wells, and Sohi ASPLOS Simulation Methodology  Virtutech SIMICS MAI running Solaris 9  CMP system: 8 out-of-order processors 2 wide, 8 stages, 128 entry ROB, 3GHz  3 level memory hierarchy Private L1 and L2 Directory base MOSI L3: Shared, Exclusive 8MB (16w) (75 cycle load-to-use) Point to point ordered interconnect (25 cycle latency) Main Memory 255 cycle load to use, 40GB/s Measure impact on predictive structures

Chakraborty, Wells, and Sohi ASPLOS L2 Instruction Reference

Chakraborty, Wells, and Sohi ASPLOS Result Summary  Branch predictors 9-25% reduction in mis-predictions  L2 data references 0-19% reduction in load misses Moderate increase in store misses  Interconnect messages Moderate reduction (after accounting extra messages for migration)

Chakraborty, Wells, and Sohi ASPLOS Performance Potential Migration Overhead

Chakraborty, Wells, and Sohi ASPLOS Talk Outline  Motivation  Computation Spreading (CSP) Case study: OS and User compution  Implementation  Results  Related Work and Summary

Chakraborty, Wells, and Sohi ASPLOS Related Work  Software re-design: staged execution Cohort Scheduling [Larus and Parkes 01], STEPS [Ailamaki 04], SEDA [Welsh 01], LARD [Pai 98] CSP: similar execution in hardware  OS and User Interference [several] Structural separation to avoid interference CSP avoids interference and exploits synergy

Chakraborty, Wells, and Sohi ASPLOS Summary  Extensive code reuse in CMPs 45-66% instruction blocks universally accessed in server workloads  Computation Spreading Localize similar computation and separate dissimilar computation Exploits core proximity in CMPs  Case Study: OS and User computation Demonstrate substantial performance potential

Chakraborty, Wells, and Sohi ASPLOS Thank You!

Chakraborty, Wells, and Sohi ASPLOS Backup Slides

Chakraborty, Wells, and Sohi ASPLOS L2 Data Reference L2 load miss comparable, slight to moderate increase in L2 store miss

Chakraborty, Wells, and Sohi ASPLOS Multiprocessor Code Reuse

Chakraborty, Wells, and Sohi ASPLOS Performance Potential