CPE 631: Multithreading: Thread-Level Parallelism Within a Processor

Slides:



Advertisements
Similar presentations
CS136, Advanced Architecture Limits to ILP Simultaneous Multithreading.
Advertisements

CS 7810 Lecture 4 Overview of Steering Algorithms, based on Dynamic Code Partitioning for Clustered Architectures R. Canal, J-M. Parcerisa, A. Gonzalez.
Multithreading processors Adapted from Bhuyan, Patterson, Eggers, probably others.
Microprocessor Microarchitecture Multithreading Lynn Choi School of Electrical Engineering.
CS 7810 Lecture 16 Simultaneous Multithreading: Maximizing On-Chip Parallelism D.M. Tullsen, S.J. Eggers, H.M. Levy Proceedings of ISCA-22 June 1995.
Single-Chip Multiprocessor Nirmal Andrews. Case for single chip multiprocessors Advances in the field of integrated chip processing. - Gate density (More.
Computer Architecture 2011 – Out-Of-Order Execution 1 Computer Architecture Out-Of-Order Execution Lihu Rappoport and Adi Yoaz.
CS 7810 Lecture 20 Initial Observations of the Simultaneous Multithreading Pentium 4 Processor N. Tuck and D.M. Tullsen Proceedings of PACT-12 September.
1 Lecture 11: SMT and Caching Basics Today: SMT, cache access basics (Sections 3.5, 5.1)
1 Lecture 11: ILP Innovations and SMT Today: out-of-order example, ILP innovations, SMT (Sections 3.5 and supplementary notes)
Multithreading and Dataflow Architectures CPSC 321 Andreas Klappenecker.
1 Lecture 12: ILP Innovations and SMT Today: ILP innovations, SMT, cache basics (Sections 3.5 and supplementary notes)
1 Lecture 10: ILP Innovations Today: ILP innovations and SMT (Section 3.5)
1 Lecture 25: Multi-core Processors Today’s topics:  Writing parallel programs  SMT  Multi-core examples Reminder:  Assignment 9 due Tuesday.
Intel Architecture. Changes in architecture Software architecture: –Front end (Feature changes such as adding more graphics, changing the background colors,
Simultaneous Multithreading: Maximizing On-Chip Parallelism Presented By: Daron Shrode Shey Liggett.
CPE 631: Multithreading: Thread-Level Parallelism Within a Processor Electrical and Computer Engineering University of Alabama in Huntsville Aleksandar.
Multi-Core Architectures
Nicolas Tjioe CSE 520 Wednesday 11/12/2008 Hyper-Threading in NetBurst Microarchitecture David Koufaty Deborah T. Marr Intel Published by the IEEE Computer.
Hardware Multithreading. Increasing CPU Performance By increasing clock frequency By increasing Instructions per Clock Minimizing memory access impact.
Hyper-Threading Technology Architecture and Micro-Architecture.
Tahir CELEBI, Istanbul, 2005 Hyper-Threading Technology Architecture and Micro-Architecture Prepared by Tahir Celebi Istanbul, 2005.
SIMULTANEOUS MULTITHREADING Ting Liu Liu Ren Hua Zhong.
Hyper-Threading Technology Architecture and Microarchitecture
Thread Level Parallelism Since ILP has inherent limitations, can we exploit multithreading? –a thread is defined as a separate process with its own instructions.
Processor Level Parallelism. Improving the Pipeline Pipelined processor – Ideal speedup = num stages – Branches / conflicts mean limited returns after.
Intel Multimedia Extensions and Hyper-Threading Michele Co CS451.
1 Lecture: SMT, Cache Hierarchies Topics: SMT processors, cache access basics and innovations (Sections B.1-B.3, 2.1)
Advanced Computer Architecture pg 1 Embedded Computer Architecture 5SAI0 Chip Multi-Processors (ch 8) Henk Corporaal
Computer Structure 2015 – Intel ® Core TM μArch 1 Computer Structure Multi-Threading Lihu Rappoport and Adi Yoaz.
1 Lecture 20: Core Design Today: Innovations for ILP, TLP, power Sign up for class presentations.
On-chip Parallelism Alvin R. Lebeck CPS 220/ECE 252.
Fall 2012 Parallel Computer Architecture Lecture 4: Multi-Core Processors Prof. Onur Mutlu Carnegie Mellon University 9/14/2012.
Pentium 4 Deeply pipelined processor supporting multiple issue with speculation and multi-threading 2004 version: 31 clock cycles from fetch to retire,
COMP 740: Computer Architecture and Implementation
Electrical and Computer Engineering
Prof. Onur Mutlu Carnegie Mellon University
Simultaneous Multithreading
Lynn Choi School of Electrical Engineering
Simultaneous Multithreading
Multi-core processors
Computer Structure Multi-Threading
INTEL HYPER THREADING TECHNOLOGY
Embedded Computer Architecture 5SAI0 Chip Multi-Processors (ch 8)
/ Computer Architecture and Design
Hyperthreading Technology
Electrical and Computer Engineering
Lecture: SMT, Cache Hierarchies
Computer Architecture: Multithreading (I)
Lecture 26: Multiprocessors
The Microarchitecture of the Pentium 4 processor
Levels of Parallelism within a Single Processor
Limits to ILP Conflicting studies of amount
Hardware Multithreading
Lecture: SMT, Cache Hierarchies
Lecture 17: Core Design Today: implementing core structures – rename, issue queue, bypass networks; innovations for high ILP and clock speed.
Ka-Ming Keung Swamy D Ponpandi
Lecture: SMT, Cache Hierarchies
Lecture 19: Core Design Today: implementing core structures – rename, issue queue, bypass networks; innovations for high ILP and clock speed.
Lecture 27: Multiprocessors
/ Computer Architecture and Design
Embedded Computer Architecture 5SAI0 Chip Multi-Processors (ch 8)
CSC3050 – Computer Architecture
Lecture: SMT, Cache Hierarchies
Levels of Parallelism within a Single Processor
Hardware Multithreading
8 – Simultaneous Multithreading
Lecture 21: Synchronization & Consistency
Lecture 22: Multithreading
Ka-Ming Keung Swamy D Ponpandi
Presentation transcript:

CPE 631: Multithreading: Thread-Level Parallelism Within a Processor Electrical and Computer Engineering University of Alabama in Huntsville Aleksandar Milenkovic milenka@ece.uah.edu http://www.ece.uah.edu/~milenka

Outline Trends in microarchitecture Exploiting thread-level parallelism Exploiting TLP within a processor Resource sharing Performance implications Design challenges Intel’s HT technology

Trends in microarchitecture Higher clock speeds To achieve high clock frequency make pipeline deeper (superpipelining) Events that disrupt pipeline (branch mispredictions, cache misses, etc) become very expensive in terms of lost clock cycles ILP: Instruction Level Parallelism Extract parallelism in a single program Superscalar processors have multiple execution units working in parallel Challenge to find enough instructions that can be executed concurrently Out-of-order execution => instructions are sent to execution units based on instruction dependencies rather than program order

Trends in microarchitecture Cache hierarchies Processor-memory speed gap Use caches to reduce memory latency Multiple levels of caches: smaller and faster closer to the processor core Thread-level Parallelism Multiple programs execute concurrently Web-servers have an abundance of software threads Users: surfing the web, listening to music, encoding/decoding video streams, etc.

Exploiting thread-level parallelism CMP – Chip Multiprocessing Multiple processors, each with a full set of architectural resources, reside on the same die Processors may share an on-chip cache or each can have its own cache Examples: HP Mako, IBM Power4 Challenges: Power, Die area (cost) Time-slice multithreading Processor switches between software threads after a predefined time slice Can minimize the effects of long lasting events Still, some execution slots are wasted

Multithreading Within a Processor Until now, we have executed multiple threads of an application on different processors – can multiple threads execute concurrently on the same processor? Why is this desireable? inexpensive – one CPU, no external interconnects no remote or coherence misses (more capacity misses) Why does this make sense? most processors can’t find enough work – peak IPC is 6, average IPC is 1.5! threads can share resources  we can increase threads without a corresponding linear increase in area

What Resources are Shared? Multiple threads are simultaneously active (in other words, a new thread can start without a context switch) For correctness, each thread needs its own PC, its own logical regs (and its own mapping from logical to phys regs) For performance, each thread could have its own ROB (so that a stall in one thread does not stall commit in other threads), I-cache, branch predictor, D-cache, etc. (for low interference), although note that more sharing  better utilization of resources Each additional thread costs a PC, rename table, and ROB – cheap!

Approaches to Multithreading Within a Processor Fine-grained multithreading: switches threads on every clock cycle Pro: hide latency of from both short and long stalls Con: Slows down execution of the individual threads ready to go Course-grained multithreading: switches threads only on costly stalls (e.g., L2 stalls) Pros: no switching each clock cycle, no slow down for ready-to-go threads Con: limitations in hiding shorter stalls Simultaneous Multithreading: exploits TLP at the same time it exploits ILP

How Resources are Shared? Each box represents an issue slot for a functional unit. Peak thruput is 4 IPC. Thread 1 Thread 2 Thread 3 Cycles Thread 4 Idle Superscalar Coarse-grained Multithreading Fine-Grained Multithreading Simultaneous Multithreading Superscalar processor has high under-utilization – not enough work every cycle, especially when there is a cache miss Fine-grained multithreading can only issue instructions from a single thread in a cycle – can not find max work every cycle, but cache misses can be tolerated Simultaneous multithreading can issue instructions from any thread every cycle – has the highest probability of finding work for every issue slot

Resource Sharing Thread-1 R1  R1 + R2 R3  R1 + R4 R5  R1 + R3 P73 P1 + P2 P74  P73 + P4 P75  P73 + P74 Instr Fetch Instr Rename Issue Queue Instr Fetch Instr Rename P73 P1 + P2 P74  P73 + P4 P75  P73 + P74 P76  P33 + P34 P77  P33 + P76 P78  P77 + P35 R2  R1 + R2 R5  R1 + R2 R3  R5 + R3 P76  P33 + P34 P77  P33 + P76 P78  P77 + P35 Thread-2 Register File FU FU FU FU

Performance Implications of SMT Single thread performance is likely to go down (caches, branch predictors, registers, etc. are shared) – this effect can be mitigated by trying to prioritize one thread While fetching instructions, thread priority can dramatically influence total throughput – a widely accepted heuristic (ICOUNT): fetch such that each thread has an equal share of processor resources With eight threads in a processor with many resources, SMT yields throughput improvements of roughly 2-4 Alpha 21464 and Intel Pentium 4 are examples of SMT

Design Challenges How many threads? Many to find enough parallelism However, mixing many threads will compromise execution of individual threads Processor front-end (instruction fetch) Fetch as far as possible in a single thread (to maximize thread performance) However, this limits the number of instructions available for scheduling from other threads Larger register files (multiple contexts) Minimize clock cycle time Cache conflicts

Pentium 4: Hyperthreading architecture One physical processor appears as multiple logical processors HT implementation on NetBurst microarchitecture has 2 logical processors Architectural State Architectural State Architectural state: general purpose registers control registers APIC: advanced programmable interrupt controller Processor execution resources

Pentium 4: Hyperthreading architecture Main processor resources are shared caches, branch predictors, execution units, buses, control logic Duplicated resources register alias tables (map the architectural registers to physical rename registers) next instruction pointer and associated control logic return stack pointer instruction streaming buffer and trace cache fill buffers

Pentium 4: Die Size and Complexity

Pentium 4: Resources sharing schemes Partition – dedicate equal resources to each logical processors Good when expect high utilization and somewhat unpredicatable Threshold – flexible resource sharing with a limit on maximum resource usage Good for small resources with bursty utilization and when the micro-ops stay in the structure for short predictable periods Full sharing – flexible with no limits Good for large structures, with variable working-set sizes

Pentium 4: Shared vs. partitioned queues Cycle 0: 2 light shaded (fast) and 2 dark shaded (slow) microoperations Cycle 1: send light shaded micro-op 0 down the pipeline In the shared queue the previous pipeline stage sends dark shaded micro-op 2, but in the partitioned queue the slow thread is already occupying two entries, so the previous stage will send light shaded micro-op 2. (c) Cycle 3: Light micro-op 1 is sent down the pipeline. Shared queue will accept light shaded micro-op 2, while the partitioned will accept the next light shaded micro-op 3. (d) Cycle 3: light shaded micro-op 2 is sent down the pipeline. In shared queue, dark shaded micro-op 3 is entering the queue, but in the partitioned we will have to accept another light shaded micro-op 4. (e) Finally, the shared queue keeps all dark shaded micro-ops (slow) and will block the other thread. With the partitioned queue we do not have such problems. shared partitioned

NetBurst Pipeline threshold partitioned

Pentium 4: Shared vs. partitioned resources E.g., major pipeline queues Threshold Puts a threshold on the number of resource entries a logical processor can have E.g., scheduler Fully shared resources E.g., caches Modest interference Benefit if we have shared code and/or data

Pentium 4: Scheduler occupancy

Pentium 4: Shared vs. partitioned cache

Pentium 4: Performance Improvements

Multi-Programmed Speedup