1. Multicore: Panic or Panacea? Mikko H. Lipasti Associate Professor Electrical and Computer Engineering University of Wisconsin – Madison http://www.ece.wisc.edu/~pharm

2. Sep 18, 2007Mikko Lipasti-University of Wisconsin Multicore Mania First, servers IBM Power4, 2001 Then desktops AMD Athlon X2, 2005 Then laptops Intel Core Duo, 2006 Soon, your cellphone ARM MPCore, prototypes for a while now

3. What is behind this trend? Moore’s Law Chip power consumption Single-thread performance trend [source: Intel] Sep 18, 2007Mikko Lipasti-University of Wisconsin

4. Dynamic Power Static CMOS: current flows when active Combinational logic evaluates new inputs Flip-flop, latch captures new value (clock edge)‏ Terms C: capacitance of circuit wire length, number and size of transistors V: supply voltage A: activity factor f: frequency Future: Fundamentally power-constrained Sep 18, 2007Mikko Lipasti-University of Wisconsin

5. Easy answer: Multicore Single CoreDual CoreQuad Core Core areaA~A/2~A/4 Core powerW~W/2~W/4 Chip powerW + OW + O’W + O’’ Core performanceP0.9P0.8P Chip performanceP1.8P3.2P Sep 18, 2007Mikko Lipasti-University of Wisconsin Core

6. f Amdahl’s Law f – fraction that can run in parallel 1-f – fraction that must run serially Sep 18, 2007Mikko Lipasti-University of Wisconsin Time # CPUs 1 1-f f n

7. Fixed Chip Power Budget Amdahl’s Law Ignores (power) cost of n cores Revised Amdahl’s Law More cores  each core is slower Parallel speedup < n Serial portion (1-f) takes longer Also, interconnect and scaling overhead Sep 18, 2007Mikko Lipasti-University of Wisconsin # CPUs Time 1 1-f f n

8. Fixed Power Scaling Fixed power budget forces slow cores Serial code quickly dominates Sep 18, 2007Mikko Lipasti-University of Wisconsin

9. Predictions and Challenges Parallel scaling limits many-core >4 cores only for well-behaved programs Optimistic about new applications Interconnect overhead Single-thread performance Will degrade unless we innovate Parallel programming Express/extract parallelism in new ways Retrain programming workforce Sep 18, 2007Mikko Lipasti-University of Wisconsin

10. Research Agenda Programming for parallelism Sources of parallelism New applications, tools, and approaches Single-thread performance and power Most attractive to programmer/user Chip multiprocessor overheads Interconnect, caches, coherence, fairness Sep 18, 2007Mikko Lipasti-University of Wisconsin

11. Finding Parallelism 1. Functional parallelism Car: {engine, brakes, entertain, nav, …} Game: {physics, logic, UI, render, …} 2. Automatic extraction [UW Multiscalar] Decompose serial programs 3. Data parallelism Vector, matrix, db table, pixels, … 4. Request parallelism Web, shared database, telephony, … Sep 18, 2007Mikko Lipasti-University of Wisconsin

12. Balancing Work Amdahl’s parallel phase f: all cores busy If not perfectly balanced (1-f) term grows (f not fully parallel) Performance scaling suffers Manageable for data & request parallel apps Very difficult problem for other two: Functional parallelism Automatically extracted Scale power to mismatch [Multiscalar] Sep 18, 2007Mikko Lipasti-University of Wisconsin

13. Coordinating Work Synchronization Some data somewhere is shared Coordinate/order updates and reads Otherwise  chaos Traditionally: locks and mutual exclusion Hard to get right, even harder to tune for perf. Research: Transactional Memory [UW Multifacet] Programmer: Declare potential conflict Hardware and/or software: speculate & check Commit or roll back and retry Sep 18, 2007Mikko Lipasti-University of Wisconsin

14. Single-thread Performance Still most attractive source of performance Speeds up parallel and serial phases Can use it to buy back power Must focus on power consumption Performance benefit ≥ Power cost Sep 18, 2007Mikko Lipasti-University of Wisconsin

15. Single-thread Performance Hardware accelerators and circuits Domain-specific [UW MESA] Reconfigurable [UW Compton] VLSI and design automation [UW WISCAD, Kursun] Increasing frequency Seems prohibitive: clock power Clever clocking schemes can help [UW Pharm] Increasing instruction-level parallelism [UW Multiscalar, UW Pharm, UW Smith] Without blowing power budget Alternatively, reduce power for same performance Sep 18, 2007Mikko Lipasti-University of Wisconsin

16. Chip Multiprocessor Overheads Core Interconnect [UW Pharm] 80% of chip power [Borkar, ISLPED ‘07 panel] Need fundamentally different approach Revisit circuit switching Cache coherence [UW Multifacet, Pharm] Match workload behavior Optimize for on-chip communication Sep 18, 2007Mikko Lipasti-University of Wisconsin

17. Chip Multiprocessor Overheads Shared caches [UW Multifacet, Multiscalar, Smith] On-chip memory can be shared Optimize replacement, replication Fairness [UW Smith] Maintain Performance isolation Share resources fairly (memory, caches) Sep 18, 2007Mikko Lipasti-University of Wisconsin

18. Research Groups @ UW Sep 18, 2007Mikko Lipasti-University of Wisconsin GroupFacultyURL ComptonKati Compton www.ece.wisc.edu/~kati KursunVolkan Kursun www.cae.wisc.edu/~kursun MESAMike Schulte mesa.ece.wisc.edu MultifacetMark Hill, David Wood http://www.cs.wisc.edu/multifacet MultiscalarGuri Sohi www.cs.wisc.edu/~mscalar PHARMMikko Lipasti www.ece.wisc.edu/~pharm SmithJames Smith www.engr.wisc.edu/ece/faculty/smith_james.html VerticalKaru Sankaralingam www.cs.wisc.edu/vertical/wiki WISCADAzadeh Davoodi www.cae.wisc.edu/~adavoodi

19. Conclusion Forecast Limited multicore (≤4) is here to stay Manycore (>4) will find its place Hardware Challenges Single-thread performance and power Multicore overhead Software Challenges Finding application parallelism Creating correct parallel programs Creating scalable parallel programs Sep 18, 2007Mikko Lipasti-University of Wisconsin

20. Questions? http://www.ece.wisc.edu/~pharm Sep 18, 2007Mikko Lipasti-University of Wisconsin