1. Multi/Many Core Systems By Ahmed Shah Mashiyat

3. Processor Trends So Far.. Smarter BrainSmarter Brain –(e.g. x386  x486  Pentium  P2  P3  P4) –(e.g. x386  x486  Pentium  P2  P3  P4) Larger MemoryLarger Memory –Larger caches, DRAM, Disk –Larger caches, DRAM, Disk Smaller HeadSmaller Head –Fewer chips (integrate more things onto a chip) –Fewer chips (integrate more things onto a chip) More Power ConsumptionMore Power Consumption –few Watts  120+ Watts! –few Watts  120+ Watts! More ComplexMore Complex –  1Billion Transistors; design + verification complexity –  1Billion Transistors; design + verification complexity

4. Processor-Memory Performance Gap µProc 60%/yr. DRAM 7%/yr. 1 10 100 1000 1980 1981 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 19941995 1996 1997 1998 1999 2000 DRAM CPU 1982 Performance “Moore’s Law” From D. Patterson, CS252, Spring 1998 ©UCB

5. Major Problem Today: End of the Road! Can’t increase Power Consumption (~100W!)Can’t increase Power Consumption (~100W!) Can’t increase Design ComplexityCan’t increase Design Complexity Can’t increase Verification RequirementsCan’t increase Verification Requirements No further improvements to a Processor?! No further improvements to a Processor?!

6. Obvious Answer: Use Multiple Brains If single brain can’t be improved, use multiple brains!If single brain can’t be improved, use multiple brains! Put multiple simple CPUs on a single chipPut multiple simple CPUs on a single chip Multi-Core! Multi-Core!

7. The First Multi-Core: IBM Power 4 Processor The First Multi-Core: IBM Power 4 Processor

8. Definitions A Multi-Core processor combines two or more independent cores into a single package composed of a single integrated circuit (IC), called a die, or more dies packaged together.A Multi-Core processor combines two or more independent cores into a single package composed of a single integrated circuit (IC), called a die, or more dies packaged together. A dual-core processor contains two cores, and a quad- core processor contains four cores. A dual-core processor contains two cores, and a quad- core processor contains four cores. Typically the term Many-Core is sometimes used to describe multi-core architectures with an especially high number of cores (tens or hundreds).Typically the term Many-Core is sometimes used to describe multi-core architectures with an especially high number of cores (tens or hundreds).

9. Multi core Single ChipSingle Chip Multiple distinct processing EngineMultiple distinct processing Engine E.g.) Shared-cache Dual Core ArchitectureE.g.) Shared-cache Dual Core Architecture Core 0 CPU L1 Cache Core 1 CPU L1 Cache L2 Cache

10. Multi core Cores in a multi-core device may share a single coherent cache at the highest on- device cache level (e.g. L2 for the Intel Core 2) or may have separate caches (e.g. current AMD dual-core processors).Cores in a multi-core device may share a single coherent cache at the highest on- device cache level (e.g. L2 for the Intel Core 2) or may have separate caches (e.g. current AMD dual-core processors). The processors also share the same interconnect to the rest of the system.The processors also share the same interconnect to the rest of the system. Each "core" independently implements optimizations such as superscalar execution, pipelining, and multithreading.Each "core" independently implements optimizations such as superscalar execution, pipelining, and multithreading. A system with n cores is effective when it is presented with n or more threads concurrently.A system with n cores is effective when it is presented with n or more threads concurrently.

11. Systems using Multi core The most commercially significant (or at least the most 'obvious') multi-core processors are those used in personal computers (primarily from Intel and AMD) and game consoles (e.g., the eight-core Cell processor in the PS3 and the three-core Xenon processor in the Xbox 360).The most commercially significant (or at least the most 'obvious') multi-core processors are those used in personal computers (primarily from Intel and AMD) and game consoles (e.g., the eight-core Cell processor in the PS3 and the three-core Xenon processor in the Xbox 360). "multi" typically means a relatively small number of cores. However, the technology is widely used in other technology areas, especially those of embedded processors, such as network processors and digital signal processors, and in GPUs."multi" typically means a relatively small number of cores. However, the technology is widely used in other technology areas, especially those of embedded processors, such as network processors and digital signal processors, and in GPUs.

12. Advantages of Multi-core The proximity of multiple CPU cores on the same die allows the cache coherency circuitry to operate at a much higher clock rate than is possible if the signals have to travel off-chip. Combining equivalent CPUs on a single die significantly improves the performance of cache snoop (alternative: Bus snooping) operations. Put simply, this means that signals between different CPUs travel shorter distances, and therefore those signals degrade less. These higher quality signals allow more data to be sent in a given time period since individual signals can be shorter and do not need to be repeated as often.The proximity of multiple CPU cores on the same die allows the cache coherency circuitry to operate at a much higher clock rate than is possible if the signals have to travel off-chip. Combining equivalent CPUs on a single die significantly improves the performance of cache snoop (alternative: Bus snooping) operations. Put simply, this means that signals between different CPUs travel shorter distances, and therefore those signals degrade less. These higher quality signals allow more data to be sent in a given time period since individual signals can be shorter and do not need to be repeated as often.

13. Advantages of Multi-core(cont..) The largest boost in performance will likely be noticed in improved response time while running CPU- intensive processes, like antivirus scans, ripping/ burning media (requiring file conversion), or searching for folders. For example, if the automatic virus scan initiates while a movie is being watched, the application running the movie is far less likely to be starved of processor power, as the antivirus program will be assigned to a different processor core than the one running the movie playback.The largest boost in performance will likely be noticed in improved response time while running CPU- intensive processes, like antivirus scans, ripping/ burning media (requiring file conversion), or searching for folders. For example, if the automatic virus scan initiates while a movie is being watched, the application running the movie is far less likely to be starved of processor power, as the antivirus program will be assigned to a different processor core than the one running the movie playback.

14. Advantages of Multi-core(cont..) Assuming that the die can fit into the package, physically, the multi-core CPU designs require much less Printed Circuit Board (PCB) space than multi-chip SMP designs. Also, a dual-core processor uses slightly less power than two coupled single-core processors, principally because of the decreased power required to drive signals external to the chip. Furthermore, the cores share some circuitry, like the L2 cache and the interface to the front side bus (FSB).Assuming that the die can fit into the package, physically, the multi-core CPU designs require much less Printed Circuit Board (PCB) space than multi-chip SMP designs. Also, a dual-core processor uses slightly less power than two coupled single-core processors, principally because of the decreased power required to drive signals external to the chip. Furthermore, the cores share some circuitry, like the L2 cache and the interface to the front side bus (FSB).

15. The Result… Today’s CPU manufacturers have turned to multi- core designs e.g. putting more than one “brain” on a single chip. Intel offers two, four or eight core chips with more to come, Sun Microsystems has chips with up to 64 cores, AMD has its quad-core chip (and others), etc. So it seems the future belongs to multi- core systems, especially in the server marketplace.Today’s CPU manufacturers have turned to multi- core designs e.g. putting more than one “brain” on a single chip. Intel offers two, four or eight core chips with more to come, Sun Microsystems has chips with up to 64 cores, AMD has its quad-core chip (and others), etc. So it seems the future belongs to multi- core systems, especially in the server marketplace.

16. A Crisis in the Industry! Parallelism is Difficult to Find!Parallelism is Difficult to Find! –Writing Parallel Programs is much harder! –Parallel Architectures, Compilers, and Programmers have been researched for many decades But, now everything will have multi-core:But, now everything will have multi-core: –Desktops, Servers, Supercomputers, Palmtops, Embedded Computers

17. Reason for the crisis… Memory bandwidthMemory bandwidth -- Memory bandwidth is a major limitation with multi-core systems. All cores must share access to the same memory and therefore have to take turns to read from or write it. Disk speedDisk speed -- Disk speed can throttle performance. If a disk can read only 40MB/sec (a common figure for even a fast laptop hard drive), then a program that needs to read a 400MB file will take a minimum of 10 seconds to open it, even if it has no computation to do.

18. Reason for the crisis… Software and scalabilitySoftware and scalability --software today is frequently not written to take advantage of multiple cores efficiently; it is technically challenging to write efficient threaded code that is free of threading bugs (threading bugs are those that result from incorrectly-written threaded code). --software today is frequently not written to take advantage of multiple cores efficiently; it is technically challenging to write efficient threaded code that is free of threading bugs (threading bugs are those that result from incorrectly-written threaded code). -- A highly scalable program is difficult to build. It requires a lot of expertise.

19. Crisis  Opportunity! Industry is willing/forced to consider:Industry is willing/forced to consider: –Code-rewrite to better expose parallelism –Newer ways of expressing parallelism –Newer programming languages –Newer architectures –Overall: Newer lines of thinking and research! Marvelous Opportunity to do Research and Development in this area!Marvelous Opportunity to do Research and Development in this area!

20. Conclusions That fancy new computer won’t deliver on its speed promise unless it offers adequate memory bandwidth for the program you actually run. Those programs need to be especially well designed to exploit more than one or two CPU cores.That fancy new computer won’t deliver on its speed promise unless it offers adequate memory bandwidth for the program you actually run. Those programs need to be especially well designed to exploit more than one or two CPU cores. So for now we are not ready to capitalize the power of multi/many core system to full extent.So for now we are not ready to capitalize the power of multi/many core system to full extent.

21. References Multi-core Computers @ http://diglloyd.com/diglloyd/free/MultiCore/index.htmlMulti-core Computers @ http://diglloyd.com/diglloyd/free/MultiCore/index.html http://diglloyd.com/diglloyd/free/MultiCore/index.html Multi-core @ http://en.wikipedia.org/wiki/Multi-coreMulti-core @ http://en.wikipedia.org/wiki/Multi-corehttp://en.wikipedia.org/wiki/Multi-core Multi-core state of play @ http://www.builderau.com.au/strategy/architecture/soa/Multi- core-state-of- play/0,339028264,339291257,00.htm?feed=pt_programmingMulti-core state of play @ http://www.builderau.com.au/strategy/architecture/soa/Multi- core-state-of- play/0,339028264,339291257,00.htm?feed=pt_programming http://www.builderau.com.au/strategy/architecture/soa/Multi- core-state-of- play/0,339028264,339291257,00.htm?feed=pt_programming http://www.builderau.com.au/strategy/architecture/soa/Multi- core-state-of- play/0,339028264,339291257,00.htm?feed=pt_programming Central processing unit @ http://en.wikipedia.org/wiki/CPUCentral processing unit @ http://en.wikipedia.org/wiki/CPU http://en.wikipedia.org/wiki/CPU Multi Core Processors: The Possibilities for a New Era in Computer Architecture… IBM Research @ http://203.197.173.12/faer/treach/pdf/Multicore-Possibilities.pdf http://203.197.173.12/faer/treach/pdf/Multicore-Possibilities.pdf HETEROGENEOUS MULTICORE PROCESSORS @ http://132.236.67.210/EngrWords/issues/ew04/Vitkalov_talk.pd f http://132.236.67.210/EngrWords/issues/ew04/Vitkalov_talk.pd f OVERVIEW OF MULTICORE, PARALLEL COMPUTING, AND DATA MINING @ http://grids.ucs.indiana.edu/ptliupages/presentations/OVERVIEW OF MULTICORE, PARALLEL COMPUTING, AND DATA MINING @ http://grids.ucs.indiana.edu/ptliupages/presentations/ http://grids.ucs.indiana.edu/ptliupages/presentations/ Introduction to High Performance Computing @ http://cse.stfx.ca/~ltyang/csci-455/Introduction to High Performance Computing @ http://cse.stfx.ca/~ltyang/csci-455/ http://cse.stfx.ca/~ltyang/csci-455/

22. THANK YOU! QUESTIONS?