Samira Khan University of Virginia Sep 13, 2017

Slides:



Advertisements
Similar presentations
ECE 454 Computer Systems Programming Compiler and Optimization (I) Ding Yuan ECE Dept., University of Toronto
Advertisements

High Performing Cache Hierarchies for Server Workloads
4/17/20151 Improving Memory Bank-Level Parallelism in the Presence of Prefetching Chang Joo Lee Veynu Narasiman Onur Mutlu* Yale N. Patt Electrical and.
Data Marshaling for Multi-Core Architectures M. Aater Suleman Onur Mutlu Jose A. Joao Khubaib Yale N. Patt.
Limits on ILP. Achieving Parallelism Techniques – Scoreboarding / Tomasulo’s Algorithm – Pipelining – Speculation – Branch Prediction But how much more.
Bottleneck Identification and Scheduling in Multithreaded Applications José A. Joao M. Aater Suleman Onur Mutlu Yale N. Patt.
Helper Threads via Virtual Multithreading on an experimental Itanium 2 processor platform. Perry H Wang et. Al.
- Sam Ganzfried - Ryan Sukauye - Aniket Ponkshe. Outline Effects of asymmetry and how to handle them Design Space Exploration for Core Architecture Accelerating.
CS 7810 Lecture 19 Coherence Decoupling: Making Use of Incoherence J.Huh, J. Chang, D. Burger, G. Sohi Proceedings of ASPLOS-XI October 2004.
1 Accelerating Critical Section Execution with Asymmetric Multi-Core Architectures M. Aater Suleman* Onur Mutlu† Moinuddin K. Qureshi‡ Yale N. Patt* *The.
Computer Organization and Architecture The CPU Structure.
Techniques for Efficient Processing in Runahead Execution Engines Onur Mutlu Hyesoon Kim Yale N. Patt.
Slipstream Processors by Pujan Joshi1 Pujan Joshi May 6 th, 2008 Slipstream Processors Improving both Performance and Fault Tolerance.
Intel Architecture. Changes in architecture Software architecture: –Front end (Feature changes such as adding more graphics, changing the background colors,
Super computers Parallel Processing By Lecturer: Aisha Dawood.
1 Computation Spreading: Employing Hardware Migration to Specialize CMP Cores On-the-fly Koushik Chakraborty Philip Wells Gurindar Sohi
Precomputation- based Prefetching By James Schatz and Bashar Gharaibeh.
Processor Structure and Function Chapter8:. CPU Structure  CPU must:  Fetch instructions –Read instruction from memory  Interpret instructions –Instruction.
Computer Architecture: Multithreading (IV) Prof. Onur Mutlu Carnegie Mellon University.
Sunpyo Hong, Hyesoon Kim
Samira Khan University of Virginia Feb 25, 2016 COMPUTER ARCHITECTURE CS 6354 Asymmetric Multi-Cores The content and concept of this course are adapted.
Fall 2012 Parallel Computer Architecture Lecture 4: Multi-Core Processors Prof. Onur Mutlu Carnegie Mellon University 9/14/2012.
Improving Multi-Core Performance Using Mixed-Cell Cache Architecture
CS 352H: Computer Systems Architecture
Memory Management.
18-447: Computer Architecture Lecture 27: Multi-Core Potpourri
Processes and threads.
18-447: Computer Architecture Lecture 30B: Multiprocessors
Threads vs. Events SEDA – An Event Model 5204 – Operating Systems.
Lecture: Large Caches, Virtual Memory
Architecting and Exploiting Asymmetry in Multi-Core Architectures
15-740/ Computer Architecture Lecture 17: Asymmetric Multi-Core
Multiscalar Processors
CSC 4250 Computer Architectures
Simultaneous Multithreading
5.2 Eleven Advanced Optimizations of Cache Performance
Cache Memory Presentation I
The University of Texas at Austin
Introduction, Focus, Overview
Lecture 14 Virtual Memory and the Alpha Memory Hierarchy
Accelerating Dependent Cache Misses with an Enhanced Memory Controller
Levels of Parallelism within a Single Processor
Hardware Multithreading
Lecture: Cache Innovations, Virtual Memory
Computer Architecture: Multithreading (IV)
15-740/ Computer Architecture Lecture 5: Precise Exceptions
CARP: Compression-Aware Replacement Policies
Adaptive Single-Chip Multiprocessing
Hyesoon Kim Onur Mutlu Jared Stark* Yale N. Patt
15-740/ Computer Architecture Lecture 14: Runahead Execution
* From AMD 1996 Publication #18522 Revision E
Prof. Onur Mutlu Carnegie Mellon University
Computer Evolution and Performance
Samira Khan University of Virginia Sep 26, 2018
15-740/ Computer Architecture Lecture 14: Prefetching
PERFORMANCE MEASURES. COMPUTATIONAL MODELS Equal Duration Model:  It is assumed that a given task can be divided into n equal subtasks, each of which.
Lecture: Cache Hierarchies
Levels of Parallelism within a Single Processor
Hardware Multithreading
CSC3050 – Computer Architecture
Patrick Akl and Andreas Moshovos AENAO Research Group
Introduction, Focus, Overview
Programming with Shared Memory Specifying parallelism
COMP60611 Fundamentals of Parallel and Distributed Systems
Lecture 18: Coherence and Synchronization
Computer Architecture Lecture 20: Heterogeneous Multi-Core Systems II
Prof. Onur Mutlu Carnegie Mellon University 9/19/2012
Lois Orosa, Rodolfo Azevedo and Onur Mutlu
ASPLOS 2009 Presented by Lara Lazier
CSC Multiprocessor Programming, Spring, 2011
Presentation transcript:

Samira Khan University of Virginia Sep 13, 2017 COMPUTER ARCHITECTURE CS 6354 Asymmetric Multi-Cores Samira Khan University of Virginia Sep 13, 2017 The content and concept of this course are adapted from CMU ECE 740

AGENDA Logistics Review from last lecture Accelerating critical sections Asymmetric Multi-Core

LOGISTICS Paper Presentation Project Proposal Due: Sep 20 (Wednesday) Sign up for slots https://docs.google.com/spreadsheets/d/1HVcGnIBhJC7DxB3sKmzPtNu9hvOoV8OrcEtFV_0aojg/edit#gid=0 You can pick other papers, but from ISCA, MICRO, ASPLOS Project Proposal Due: Sep 20 (Wednesday) Start early Read the related work – talk about those in the proposal Problem, novelty, key ideas, experiments, detailed plan 2-3 students per group https://docs.google.com/spreadsheets/d/1DkqIoJ623mDtuLa2AD-9DG5mVPgF4RDLgjyce1AiiW4/edit#gid=0

Asymmetric Chip Multiprocessor (ACMP) Provide one large core and many small cores + Accelerate serial part using the large core (2 units) + Execute parallel part on small cores and large core for high throughput (12+2 units) Large core Large core “Tile-Large” Small core “Tile-Small” Small core Large core ACMP

ACCELERATING PARALLEL BOTTLENECKS Serialized or imbalanced execution in the parallel portion can also benefit from a large core Examples: Critical sections that are contended Parallel stages that take longer than others to execute Idea: Dynamically identify these code portions that cause serialization and execute them on a large core

ACCELERATED CRITICAL SECTIONS (ACS) Small Core Small Core Large Core A = compute() A = compute() LOCK X result = CS(A) UNLOCK X print result PUSH A CSCALL X, Target PC … CSCALL Request Send X, TPC, STACK_PTR, CORE_ID … Waiting in Critical Section Request Buffer (CSRB) … Acquire X POP A result = CS(A) PUSH result Release X CSRET X TPC: CSDONE Response POP result print result Suleman et al., “Accelerating Critical Section Execution with Asymmetric Multi-Core Architectures,” ASPLOS 2009.

ACS PERFORMANCE TRADEOFFS Pluses + Faster critical section execution + Shared locks stay in one place: better lock locality + Shared data stays in large core’s (large) caches: better shared data locality, less ping-ponging Minuses - Large core dedicated for critical sections: reduced parallel throughput - CSCALL and CSDONE control transfer overhead - Thread-private data needs to be transferred to large core: worse private data locality

ACS PERFORMANCE TRADEOFFS Fewer parallel threads vs. accelerated critical sections Accelerating critical sections offsets loss in throughput As the number of cores (threads) on chip increase: Fractional loss in parallel performance decreases Increased contention for critical sections makes acceleration more beneficial Overhead of CSCALL/CSDONE vs. better lock locality ACS avoids “ping-ponging” of locks among caches by keeping them at the large core More cache misses for private data vs. fewer misses for shared data ACS dedicates the large core for execution of critical sections which could otherwise be used for executing additional threads. This reduces peak parallel throughput. However, we find that in critical section intensive workloads this is not a problem since the benefit obtained by accelerating of critical sections offsets the loss in peak parallel throughput. Moreover, we observe that this problem will be further reduce as the number of cores on the chip increase for two reasons. First, the fraction of throughput lost due to ACS decreases. For example, loosing 4 cores in a 64-core system will be a much smaller loss than loosing 4 cores in a 8-core system. Second, contention for critical sections increases as the number of concurrent threads increase. When contention is high, accelerating the critical sections not only reduces the critical section execution time BUT also the waiting time for contending threads. Thereby making the acceleration even more beneficial. ACS also incurs the overhead of sending CSCALL and CSDONE signals. This overhead is similar to the conventional systems because in conventional systems lock acquire operations often generate a cache miss and the lock variable is brought from another core. In ACS, since all critical sections execute on one LARGE core, the lock variables stay resident in its cache which saves cache misses. Thus, overall, ACS has similar latency. In ACS the input arguments to the critical section must be transferred from the cache of the small core to the cache of the large core. Let me explain this trade-off with an example.

CACHE MISSES FOR PRIVATE DATA PriorityHeap.insert (NewSubProblems) Shared Data: The priority heap Private Data: NewSubProblems Consider this critical section from the puzzle benchmark. This critical sections protects a priority heap. The input argument is the node to be inserted on the heap. The priority heap is the Shared data which is data protected by the critical sections and private data is the incoming node to be inserted. During execution, multiple nodes of the heap are touched to find the right place to insert the incoming private data. In conventional systems, the shared data usually moves from cache-to-cache as different cores modify it inside critical sections. The private data is usually available locally. In ACS, since all critical sections execute on the large core, the shared data stays resident AT the large and does not move from cache to cache. However, private data has to be brought in from the small requesting core to execute the critical section. Puzzle Benchmark

ACS PERFORMANCE TRADEOFFS Fewer parallel threads vs. accelerated critical sections Accelerating critical sections offsets loss in throughput As the number of cores (threads) on chip increase: Fractional loss in parallel performance decreases Increased contention for critical sections makes acceleration more beneficial Overhead of CSCALL/CSDONE vs. better lock locality ACS avoids “ping-ponging” of locks among caches by keeping them at the large core More cache misses for private data vs. fewer misses for shared data Cache misses reduce if shared data > private data This problem can be solved

ACS COMPARISON POINTS SCMP ACMP ACS Conventional locking Small core Small core SCMP Small core Large core ACMP Small core Large core ACS In our experiments we simulated three configurations. First, A symmetric CMP or SCMP with all small cores. The numbers of cores is equal to the chip area and conventional locks are used for critical sections. Second, an ACMP with one large core and remaining small cores. The large core takes the area of 4 small cores. The large core is used to execute Amdahl’s bottleneck. Critical Sections are executed using conventional locking. Third, ACS, which is an ACMP with a CSRB and support for accelerating critical sections. In ACS, the Amdahl’s bottleneck as well as the critical sections execute on the large core. The large core replceas four small cores. When chip area increases, we increase the number of small cores in all three confgiurations. At area 16, the SCMP has 16 small cores and the ACMP and ACS has 1 large core and 12 small cores. At area 32, the SCMP has 32 small cores and ACMP and ACS have 1 large core and 28 small cores. We use the ACMP as our baseline. Conventional locking Conventional locking Large core executes Amdahl’s serial part Large core executes Amdahl’s serial part and critical sections

ACCELERATED CRITICAL SECTIONS: METHODOLOGY Workloads: 12 critical section intensive applications Data mining kernels, sorting, database, web, networking Multi-core x86 simulator 1 large and 28 small cores Aggressive stream prefetcher employed at each core Details: Large core: 2GHz, out-of-order, 128-entry ROB, 4-wide, 12-stage Small core: 2GHz, in-order, 2-wide, 5-stage Private 32 KB L1, private 256KB L2, 8MB shared L3 On-chip interconnect: Bi-directional ring, 5-cycle hop latency

ACS PERFORMANCE Equal-area comparison Number of threads = Best threads Chip Area = 32 small cores SCMP = 32 small cores ACMP = 1 large and 28 small cores Equal-area comparison Number of threads = Best threads 269 180 185 Coarse-grain locks Fine-grain locks

EQUAL-AREA COMPARISONS ------ SCMP ------ ACMP ------ ACS EQUAL-AREA COMPARISONS Number of threads = No. of cores Speedup over a small core (a) ep (b) is (c) pagemine (d) puzzle (e) qsort (f) tsp Now we will compare SCMP, ACMP, and ACS as the chip area increases. The X-axis is chip area and the Y-axis shows speedup over a SINGLE small core. The green line shows ACS, the red line shows the ACMP, and blue line shows the SCMP. Here we set the number of threads equal to the number of available cores. As you can see, critical sections severely limit the scalability of some benchmarks. For example, performance of PageMine saturates at only 8 threads. Notice that the peak speedup of ACS is higher than both ACMP and SCMP and ACS does not saturate until 12 threads. More importantly, In case of puzzle and oltp-1, while the ACMP and SCMP show poor scalability ACS significantly improves scalability as well as speedup. In all, ACS improves scalability in 7 out of the 12 workloads. (g) sqlite (h) iplookup (i) oltp-1 (i) oltp-2 (k) specjbb (l) webcache Chip Area (small cores)

ACS SUMMARY Critical sections reduce performance and limit scalability Accelerate critical sections by executing them on a powerful core ACS reduces average execution time by: 34% compared to an equal-area SCMP 23% compared to an equal-area ACMP ACS improves scalability of 7 of the 12 workloads Generalizing the idea: Accelerate all bottlenecks (“critical paths”) by executing them on a powerful core

USES OF ASYMMETRY So far: What else can we do with asymmetry? Improvement in serial performance (sequential bottleneck) What else can we do with asymmetry? Energy reduction? Energy/performance tradeoff? Improvement in parallel portion?

USES OF CMPs Can you think about using these ideas to improve single-threaded performance? Implicit parallelization: thread level speculation Slipstream processors Leader-follower architectures Helper threading Prefetching Branch prediction Exception handling Redundant execution to tolerate soft (and hard?) errors

SLIPSTREAM PROCESSORS Goal: use multiple hardware contexts to speed up single thread execution (implicitly parallelize the program) Idea: Divide program execution into two threads: Advanced thread executes a reduced instruction stream, speculatively Redundant thread uses results, prefetches, predictions generated by advanced thread and ensures correctness Benefit: Execution time of the overall program reduces Core idea is similar to many thread-level speculation approaches, except with a reduced instruction stream Sundaramoorthy et al., “Slipstream Processors: Improving both Performance and Fault Tolerance,” ASPLOS 2000.

SLIPSTREAMING “At speeds in excess of 190 m.p.h., high air pressure forms at the front of a race car and a partial vacuum forms behind it. This creates drag and limits the car’s top speed. A second car can position itself close behind the first (a process called slipstreaming or drafting). This fills the vacuum behind the lead car, reducing its drag. And the trailing car now has less wind resistance in front (and by some accounts, the vacuum behind the lead car actually helps pull the trailing car). As a result, both cars speed up by several m.p.h.: the two combined go faster than either can alone.”

SLIPSTREAM PROCESSORS Detect and remove ineffectual instructions; run a shortened “effectual” version of the program (Advanced or A-stream) in one thread context Ensure correctness by running a complete version of the program (Redundant or R-stream) in another thread context Shortened A-stream runs fast; R-stream consumes near-perfect control and data flow outcomes from A-stream and finishes close behind Two streams together lead to faster execution (by helping each other) than a single one alone

SLIPSTREAM IDEA AND POSSIBLE HARDWARE

INSTRUCTION REMOVAL IN SLIPSTREAM IR detector Monitors retired R-stream instructions Detects ineffectual instructions and conveys them to the IR predictor Ineffectual instruction examples: dynamic instructions that repeatedly and predictably have no observable effect (e.g., unreferenced writes, non-modifying writes) dynamic branches whose outcomes are consistently predicted correctly. IR predictor Removes an instruction from A-stream after repeated indications from the IR detector A stream skips ineffectual instructions, executes everything else and inserts their results into delay buffer R stream executes all instructions but uses results from the delay buffer as predictions

WHAT IF A-STREAM DEVIATES FROM CORRECT EXECUTION? Why A-stream deviates due to incorrect removal or stale data access in L1 data cache How to detect it? Branch or value misprediction happens in R-stream (known as an IR misprediction) How to recover? Restore A-stream register state: copy values from R-stream registers using delay buffer or shared-memory exception handler Restore A-stream memory state: invalidate A-stream L1 data cache (or speculatively written blocks by A-stream)

Slipstream Questions How to construct the advanced thread Original proposal: Dynamically eliminate redundant instructions (silent stores, dynamically dead instructions) Dynamically eliminate easy-to-predict branches Other ways: Dynamically ignore long-latency stalls Static based on profiling How to speed up the redundant thread Original proposal: Reuse instruction results (control and data flow outcomes from the A-stream) Other ways: Only use branch results and prefetched data as predictions

DUAL CORE EXECUTION Idea: One thread context speculatively runs ahead on load misses and prefetches data for another thread context Zhou, “Dual-Core Execution: Building a Highly Scalable Single- Thread Instruction Window,” PACT 2005.

DUAL CORE EXECUTION: FRONT PROCESSOR The front processor runs faster by invalidating long-latency cache-missing loads, same as runahead execution Load misses and their dependents are invalidated Branch mispredictions dependent on cache misses cannot be resolved Highly accurate execution as independent operations are not affected Accurate prefetches to warm up caches Correctly resolved independent branch mispredictions

DUAL CORE EXECUTION: BACK PROCESSOR Re-execution ensures correctness and provides precise program state Resolve branch mispredictions dependent on long-latency cache misses Back processor makes faster progress with help from the front processor Highly accurate instruction stream Warmed up data caches

Dual Core Execution

DCE MICROARCHITECTURE

DUAL CORE EXECUTION VS. SLIPSTREAM Dual-core execution does not remove dead instructions reuse instruction register results uses the “leading” hardware context solely for prefetching and branch prediction + Easier to implement, smaller hardware cost and complexity - “Leading thread” cannot run ahead as much as in slipstream when there are no cache misses - Not reusing results in the “trailing thread” can reduce overall performance benefit

SOME RESULTS

HETEROGENEITY (ASYMMETRY)  SPECIALIZATION Heterogeneity and asymmetry have the same meaning Contrast with homogeneity and symmetry Heterogeneity is a very general system design concept (and life concept, as well) Idea: Instead of having multiple instances of the same “resource” to be the same (i.e., homogeneous or symmetric), design some instances to be different (i.e., heterogeneous or asymmetric) Different instances can be optimized to be more efficient in executing different types of workloads or satisfying different requirements/goals Heterogeneity enables specialization/customization

WHY ASYMMETRY IN DESIGN? (I) Different workloads executing in a system can have different behavior Different applications can have different behavior Different execution phases of an application can have different behavior The same application executing at different times can have different behavior (due to input set changes and dynamic events) E.g., locality, predictability of branches, instruction-level parallelism, data dependencies, serial fraction, bottlenecks in parallel portion, interference characteristics, … Systems are designed to satisfy different metrics at the same time There is almost never a single goal in design, depending on design point E.g., Performance, energy efficiency, fairness, predictability, reliability, availability, cost, memory capacity, latency, bandwidth, …

WHY ASYMMETRY IN DESIGN? (II) Problem: Symmetric design is one-size-fits-all It tries to fit a single-size design to all workloads and metrics It is very difficult to come up with a single design that satisfies all workloads even for a single metric that satisfies all design metrics at the same time This holds true for different system components, or resources Cores, caches, memory, controllers, interconnect, disks, servers, … Algorithms, policies, …

Samira Khan University of Virginia Sep 13, 2017 COMPUTER ARCHITECTURE CS 6354 Asymmetric Multi-Cores Samira Khan University of Virginia Sep 13, 2017 The content and concept of this course are adapted from CMU ECE 740