Lec 10 – Example Architectures - Simultaneous Multithreading

Slides:



Advertisements
Similar presentations
CS136, Advanced Architecture Limits to ILP Simultaneous Multithreading.
Advertisements

CPE 731 Advanced Computer Architecture ILP: Part V – Multiple Issue Dr. Gheith Abandah Adapted from the slides of Prof. David Patterson, University of.
Multithreading processors Adapted from Bhuyan, Patterson, Eggers, probably others.
Microprocessor Microarchitecture Multithreading Lynn Choi School of Electrical Engineering.
1 Digital Equipment Corporation (DEC) PDP-1. Spacewar (below), the first video game was developed on a PDP-1. DEC PDP-8. Extremely successful. Made DEC.
CS252 Graduate Computer Architecture Lecture 11 Limits to ILP / Multithreading March 1 st, 2010 John Kubiatowicz Electrical Engineering and Computer Sciences.
Limits of Instruction-Level Parallelism CS 282 – KAUST – Spring 2010 Muhamed Mudawar Original slides created by: David Patterson.
1 Pipelining for Multi- Core Architectures. 2 Multi-Core Technology Single Core Dual CoreMulti-Core + Cache + Cache Core 4 or more cores.
1 Lecture 10: ILP Innovations Today: ILP innovations and SMT (Section 3.5)
CSE 820 Graduate Computer Architecture Lec 9 – Limits to ILP and Simultaneous Multithreading Base on slides by David Patterson.
CPE 731 Advanced Computer Architecture Thread Level Parallelism Dr. Gheith Abandah Adapted from the slides of Prof. David Patterson, University of California,
CSE 502 Graduate Computer Architecture Lec 10 –Simultaneous Multithreading Larry Wittie Computer Science, StonyBrook University
8 – Simultaneous Multithreading. 2 Review from Last Time Limits to ILP (power efficiency, compilers, dependencies …) seem to limit to 3 to 6 issue for.
Simultaneous Multithreading: Maximizing On-Chip Parallelism Presented By: Daron Shrode Shey Liggett.
Limits to ILP How much ILP is available using existing mechanisms with increasing HW budgets? Do we need to invent new HW/SW mechanisms to keep on processor.
POLITECNICO DI MILANO Parallelism in wonderland: are you ready to see how deep the rabbit hole goes? Multithreaded and multicore processors Marco D. Santambrogio:
High Performance Computing Processors Felix Noble Mirayma V. Rodriguez Agnes Velez Electric and Computer Engineer Department August 25, 2004.
Larry Wittie Computer Science, StonyBrook University and ~lw
Comparing Intel’s Core with AMD's K8 Microarchitecture IS 3313 December 14 th.
Computer Architecture Lec 10 –Simultaneous Multithreading.
Thread Level Parallelism Since ILP has inherent limitations, can we exploit multithreading? –a thread is defined as a separate process with its own instructions.
Processor Level Parallelism. Improving the Pipeline Pipelined processor – Ideal speedup = num stages – Branches / conflicts mean limited returns after.
Csci 211 Computer System Architecture Limits on ILP and Simultaneous Multithreading Xiuzhen Cheng Department of Computer Sciences The George Washington.
EECS 252 Graduate Computer Architecture Lec 9 – Limits to ILP and Simultaneous Multithreading David Patterson Electrical Engineering and Computer Sciences.
Chapter 3.4: Loop-Level Parallelism and Thread-Level Parallelism
CSC 7080 Graduate Computer Architecture Lec 8 – Multiprocessors & Thread- Level Parallelism (3) – Sun T1 Dr. Khalaf Notes adapted from: David Patterson.
CSC 7080 Graduate Computer Architecture Lec 6 – Limits to ILP and Simultaneous Multithreading Dr. Khalaf Notes adapted from: David Patterson Electrical.
Advanced Computer Architecture pg 1 Embedded Computer Architecture 5SAI0 Chip Multi-Processors (ch 8) Henk Corporaal
On-chip Parallelism Alvin R. Lebeck CPS 220/ECE 252.
1 Chapter 3: Limits on ILP Limits to ILP (another perspective) Thread Level Parallelism Multithreading Simultaneous Multithreading Power 4 vs. Power 5.
Advanced Computer Architecture 5MD00 / 5Z033 SMT Simultaneously Multi-Threading Henk Corporaal TUEindhoven.
CS203 – Advanced Computer Architecture TLP – Multithreaded Architectures.
Ch3. Limits on Instruction-Level Parallelism 1. ILP Limits 2. SMT (Simultaneous Multithreading) ECE562/468 Advanced Computer Architecture Prof. Honggang.
Pentium 4 Deeply pipelined processor supporting multiple issue with speculation and multi-threading 2004 version: 31 clock cycles from fetch to retire,
COMP 740: Computer Architecture and Implementation
CS203 – Advanced Computer Architecture
Limits to ILP How much ILP is available using existing mechanisms with increasing HW budgets? Do we need to invent new HW/SW mechanisms to keep on processor.
/ Computer Architecture and Design
CPE 731 Advanced Computer Architecture Thread Level Parallelism
CPE 731 Advanced Computer Architecture ILP: Part V – Multiple Issue
Simultaneous Multithreading
Lynn Choi School of Electrical Engineering
Multi-core processors
Computer Structure Multi-Threading
Limits on ILP and Multithreading
CC 423: Advanced Computer Architecture Limits to ILP
Chapter 3: ILP and Its Exploitation
Embedded Computer Architecture 5SAI0 Chip Multi-Processors (ch 8)
/ Computer Architecture and Design
David Patterson Electrical Engineering and Computer Sciences
Advanced Computer Architecture 5MD00 / 5Z033 SMT Simultaneously Multi-Threading Henk Corporaal TUEindhoven.
John Kubiatowicz Electrical Engineering and Computer Sciences
Flow Path Model of Superscalars
Hyperthreading Technology
Electrical and Computer Engineering
David Patterson Electrical Engineering and Computer Sciences
Advanced Computer Architecture 5MD00 / 5Z033 SMT Simultaneously Multi-Threading Henk Corporaal TUEindhoven.
Lecture on High Performance Processor Architecture (CS05162)
Lecture 8: ILP and Speculation Contd. Chapter 2, Sections 2. 6, 2
Limits to ILP Conflicting studies of amount
Lec 9 – Limits to ILP and Simultaneous Multithreading
Hardware Multithreading
Comparison of Two Processors
CPE 631: Multithreading: Thread-Level Parallelism Within a Processor
CC423: Advanced Computer Architecture ILP: Part V – Multiple Issue
/ Computer Architecture and Design
Embedded Computer Architecture 5SAI0 Chip Multi-Processors (ch 8)
8 – Simultaneous Multithreading
Advanced Computer Architecture 5MD00 / 5Z032 SMT Simultaneously Multi-Threading Henk Corporaal TUEindhoven.
Chapter 3: Limits of Instruction-Level Parallelism
Presentation transcript:

Lec 10 – Example Architectures - Simultaneous Multithreading

Simultaneous Multithreading (SMT) Simultaneous multithreading (SMT): insight that dynamically scheduled processor already has many HW mechanisms to support multithreading Large set of virtual registers that can be used to hold the register sets of independent threads Register renaming provides unique register identifiers, so instructions from multiple threads can be mixed in datapath without confusing sources and destinations across threads Out-of-order completion allows the threads to execute out of order, and get better utilization of the HW Just adding a per thread renaming table and keeping separate PCs Independent commitment can be supported by logically keeping a separate reorder buffer for each thread Source: Micrprocessor Report, December 6, 1999 “Compaq Chooses SMT for Alpha” 9/18/2018

Multithreaded Categories Simultaneous Multithreading Superscalar Fine-Grained Coarse-Grained Multiprocessing Time (processor cycle) Thread 1 Thread 3 Thread 5 Thread 2 Thread 4 Idle slot 9/18/2018

Design Challenges in SMT Since SMT makes sense only with fine-grained implementation, impact of fine-grained scheduling on single thread performance? A preferred thread approach sacrifices neither throughput nor single-thread performance? Unfortunately, with a preferred thread, the processor is likely to sacrifice some throughput, when preferred thread stalls Larger register file needed to hold multiple contexts Not affecting clock cycle time, especially in Instruction issue - more candidate instructions need to be considered Instruction completion - choosing which instructions to commit may be challenging Ensuring that cache and TLB conflicts generated by SMT do not degrade performance 9/18/2018

Simultaneous Multithreaded Processor

Intel Pentium-4 Xeon Processor Hyperthreading == SMT Dual physical processors, each 2-way SMT Logical processors share nearly all resources of the physical processor Caches, execution units, branch predictors Die area overhead of hyperthreading ~5 % When one logical processor is stalled, the other can make progress No logical processor can use all entries in queues when two threads are active A processor running only one active software thread to run at the same speed with or without hyperthreading

Intel Xeon Performance

Power 4 Power 5 2 fetch (PC), 2 initial decodes 2 commits (architected register sets) Power 5 2 fetch (PC), 2 initial decodes

Power 5 data flow ... Why only 2 threads? With 4, one of the shared resources (physical registers, cache, memory bandwidth) would be prone to bottleneck

Changes in Power 5 to support SMT Increased associativity of L1 instruction cache and the instruction address translation buffers Added per thread load and store queues Increased size of the L2 (1.92 vs. 1.44 MB) and L3 caches Added separate instruction prefetch and buffering per thread Increased the number of virtual registers from 152 to 240 Increased the size of several issue queues The Power5 core is about 24% larger than the Power4 core because of the addition of SMT support 9/18/2018

Initial Performance of SMT Pentium 4 Extreme SMT yields 1.01 speedup for SPECint_rate benchmark and 1.07 for SPECfp_rate Pentium 4 is dual threaded SMT SPECRate requires that each SPEC benchmark be run against a vendor-selected number of copies of the same benchmark Running on Pentium 4 each of 26 SPEC benchmarks paired with every other (262 runs) speed-ups from 0.90 to 1.58; average was 1.20 Power 5, 8 processor server 1.23 faster for SPECint_rate with SMT, 1.16 faster for SPECfp_rate Power 5 running 2 copies of each app speedup between 0.89 and 1.41 Most gained some Fl.Pt. apps had most cache conflicts and least gains 9/18/2018

Head to Head ILP competition Processor Micro architecture Fetch / Issue / Execute FU Clock Rate (GHz) Transis-tors Die size Power Intel Pentium 4 Extreme Speculative dynamically scheduled; deeply pipelined; SMT 3/3/4 7 int. 1 FP 3.8 125 M 122 mm2 115 W AMD Athlon 64 FX-57 Speculative dynamically scheduled 6 int. 3 FP 2.8 114 M 115 mm2 104 W IBM Power5 (1 CPU only) Speculative dynamically scheduled; SMT; 2 CPU cores/chip 8/4/8 6 int. 2 FP 1.9 200 M 300 mm2 (est.) 80W (est.) Intel Itanium 2 Statically scheduled VLIW-style 6/5/11 9 int. 2 FP 1.6 592 M 423 mm2 130 W 9/18/2018

Performance on SPECint2000 9/18/2018

Performance on SPECfp2000 9/18/2018

Normalized Performance: Efficiency Rank I tanium2 Pen t I um4 A h l on Powe r 5 Int/Trans 4 2 1 3 FP/Trans Int/area FP/area Int/Watt FP/Watt 9/18/2018

No Silver Bullet for ILP No obvious over all leader in performance The AMD Athlon leads on SPECInt performance followed by the Pentium 4, Itanium 2, and Power5 Itanium 2 and Power5, which perform similarly on SPECFP, clearly dominate the Athlon and Pentium 4 on SPECFP Itanium 2 is the most inefficient processor both for Fl. Pt. and integer code for all but one efficiency measure (SPECFP/Watt) Athlon and Pentium 4 both make good use of transistors and area in terms of efficiency, IBM Power5 is the most effective user of energy on SPECFP and essentially tied on SPECINT 9/18/2018

Limits to ILP Doubling issue rates above today’s 3-6 instructions per clock, say to 6 to 12 instructions, probably requires a processor to issue 3 or 4 data memory accesses per cycle, resolve 2 or 3 branches per cycle, rename and access more than 20 registers per cycle, and fetch 12 to 24 instructions per cycle. The complexities of implementing these capabilities is likely to mean sacrifices in the maximum clock rate E.g, widest issue processor is the Itanium 2, but it also has the slowest clock rate, despite the fact that it consumes the most power! 9/18/2018

Limits to ILP Most techniques for increasing performance increase power consumption The key question is whether a technique is energy efficient: does it increase power consumption faster than it increases performance? Multiple issue processors techniques all are energy inefficient: Issuing multiple instructions incurs some overhead in logic that grows faster than the issue rate grows Growing gap between peak issue rates and sustained performance Number of transistors switching = f(peak issue rate), and performance = f( sustained rate), growing gap between peak and sustained performance  increasing energy per unit of performance 9/18/2018

Commentary Itanium architecture does not represent a significant breakthrough in scaling ILP or in avoiding the problems of complexity and power consumption Instead of pursuing more ILP, architects are increasingly focusing on TLP implemented with single-chip multiprocessors In 2000, IBM announced the 1st commercial single-chip, general-purpose multiprocessor, the Power4, which contains 2 Power3 processors and an integrated L2 cache Since then, Sun Microsystems, AMD, and Intel have switch to a focus on single-chip multiprocessors rather than more aggressive uniprocessors. Right balance of ILP and TLP is unclear today Perhaps right choice for server market, which can exploit more TLP, may differ from desktop, where single-thread performance may continue to be a primary requirement 9/18/2018

And in conclusion … Limits to ILP (power efficiency, compilers, dependencies …) seem to limit to 3 to 6 issue for practical options Explicitly parallel (Data level parallelism or Thread level parallelism) is next step to performance Coarse grain vs. Fine grained multihreading Only on big stall vs. every clock cycle Simultaneous Multithreading if fine grained multithreading based on OOO superscalar microarchitecture Instead of replicating registers, reuse rename registers Itanium/EPIC/VLIW is not a breakthrough in ILP Balance of ILP and TLP decided in marketplace 9/18/2018

Multicore Multithreaded Architecture Examples: SUN Niagra Intel Neahlem Network Processors 9/18/2018 CS252 S06 Lec9 Limits and SMT

Intel Core 2 Duo Introduction Motivation A Multi-Core on our desks A new microarchitecture to replace Netburst Intel Core 2 Duo A dual-core CPU ISA with SIMD Extension Intel Core microarchitecture Memory Hierarchy System

Core 2 Duo Microarchitecture The Cores Single-die(107 mm²), Two identical core(L1 cache 64K x 2), Shared L2 cache 6M No Hyper-threading, no L3 cache Keep front-side bus Larger L2 cache

Microarchitecture • 14-stage Pipeline • 4 wide decode • 4 wide Retire • Macro-fusion • Enhanced ALUs • Deeper Buffers Each Core CPU fetchs 128 bits of instructions, decodes 4+1 x86 instructions, issues 7 uops, reorders and renames 4 uops, dispatches 6 uops to execution units and retires up to 4 uops each cycle

Intel Clovertown Machine – 2 Quad-core Xeon Processor Multi-Core Architecture Increased resource density Major Challenges Parallelism in legacy program scalability Current solutions Software Naïve multithreading Hardware Receive Side Scaling (RSS) in NIC No Hyperthreading

Nehalem Architecture – 2-Way SMT

Nehalem – Enhanced Core 2 Datapath

T1 (“Niagara”) Target: Commercial server applications High thread level parallelism (TLP) Large numbers of parallel client requests Low instruction level parallelism (ILP) High cache miss rates Many unpredictable branches Frequent load-load dependencies Power, cooling, and space are major concerns for data centers Metric: Performance/Watt/Sq. Ft. Approach: Multicore, Fine-grain multithreading, Simple pipeline, Small L1 caches, Shared L2

T1 Architecture Also ships with 6 or 4 processors

T1 pipeline Hazards: Data Structural Single issue, in-order, 6-deep pipeline: F, S, D, E, M, W 3 clock delays for loads & branches. Shared units: L1 $, L2 $ TLB X units pipe registers Hazards: Data Structural

T1 Fine-Grained Multithreading Each core supports four threads and has its own level one caches (16KB for instructions and 8 KB for data) Switching to a new thread on each clock cycle Idle threads are bypassed in the scheduling Waiting due to a pipeline delay or cache miss Processor is idle only when all 4 threads are idle or stalled Both loads and branches incur a 3 cycle delay that can only be hidden by other threads A single set of floating point functional units is shared by all 8 cores floating point performance was not a focus for T1

Microprocessor Comparison SUN T1 Opteron Pentium D IBM Power 5 Cores 8 2 Instruction issues / clock / core 1 3 4 Peak instr. issues / chip 6 Multithreading Fine-grained No SMT L1 I/D in KB per core 16/8 64/64 12K uops/16 64/32 L2 per core/shared 3 MB shared 1MB / core 1MB/ core 1.9 MB shared Clock rate (GHz) 1.2 2.4 3.2 1.9 Transistor count (M) 300 233 230 276 Die size (mm2) 379 199 206 389 Power (W) 79 110 130 125 9/18/2018

Performance Relative to Pentium D 9/18/2018

Performance/mm2, Performance/Watt 9/18/2018

Niagara 2 Improve performance by increasing threads supported per chip from 32 to 64 8 cores * 8 threads per core Floating-point unit for each core, not for each chip Hardware support for encryption standards EAS, 3DES, and elliptical-curve cryptography Niagara 2 will add a number of 8x PCI Express interfaces directly into the chip in addition to integrated 10Gigabit Ethernet XAU interfaces and Gigabit Ethernet ports. Integrated memory controllers will shift support from DDR2 to FB-DIMMs and double the maximum amount of system memory.

SUN Niagara 2 Characteristics of Sun T5120 Niagara 2 2 pipelines per core 4 hardware threads per pipeline (b) The bars show the average utilization (%) at each level in the core; the lines represent the range of peak high and peak low values (%).

9/18/2018

9/18/2018

9/18/2018