1. SyNAR: Systems Networking and Architecture Group Symbiotic Jobscheduling for a Simultaneous Multithreading Processor Presenter: Alexandra Fedorova Simon Fraser University Authors: Allan Snavely and Dean Tullsen

2. SyNAR: Systems Networking and Architecture Group Super-scalar Processor time Issue slots Super-scalar processor has multiple issue slots A “ slot ” means we can issue/schedule an instruction Many issue slots  many instructions issued per cycle This is possible, because we have many functional units

3. SyNAR: Systems Networking and Architecture Group Problem: Under-Utilization time Issue slots A single thread is not always able to fill all the slots Slots are left unused – we waste energy! One solution is speculative, out-of-order execution, but it is difficult to implement and has limitations.

4. SyNAR: Systems Networking and Architecture Group Simultaneous Multithreading time Issue slots An idea: fill unused slots with instructions from multiple threads More instructions to choose from – more opportunity to fill the issue slots!

5. SyNAR: Systems Networking and Architecture Group Problem: Contention Resource sharing increases utilization, but also LEADS TO CONTENTION

6. SyNAR: Systems Networking and Architecture Group Research Question How to enable sharing of processor hardware without causing contention?

7. SyNAR: Systems Networking and Architecture Group Outline Background and Problem Statement Overview of the Idea Challenge Details of the Solution Research Methodology Results

8. SyNAR: Systems Networking and Architecture Group Idea: Symbiotic Schedules Assumption: OS has a queue of threads ready to execute. Some threads compete less than others Co-schedule threads that complete less CPU Scheduling queue

9. SyNAR: Systems Networking and Architecture Group Challenge How do we measure the degree of contention? How do we identify co-schedules that have little contention? ?

10. SyNAR: Systems Networking and Architecture Group Measuring Contention Background: IPC = instructions/cycle (measure of progress) IPC SMT – thread’s IPC on SMT processor: more contention  lower IPC SMT IPC single – thread’s IPC running alone IPC SMT / IPC single – measure of symbiosis for a given thread

11. SyNAR: Systems Networking and Architecture Group Weighted Speedup Sum of symbiosis measures across all N threads:

12. SyNAR: Systems Networking and Architecture Group Maximizing Symbiosis How to achieve the best symbiosis online? Proposal #1: – Run different thread combos – Measure their Weighted Speedup – Remember combos with the best Weighted Speedup – Co-schedule them in the future. Problem? Cannot measure WS online!!!

13. SyNAR: Systems Networking and Architecture Group Problem: Predicting WS WS cannot be measured online – Only offline, in lab conditions So we must estimate it – using metrics available online

14. SyNAR: Systems Networking and Architecture Group Estimating WS Obtain hardware performance metrics (available online) Measure WS (available offline ) Observe correlation between metrics and WS Build a model to predict WS

15. SyNAR: Systems Networking and Architecture Group Estimating WS: Part I 1. Measure IPC SMT 2. Measure IPC single Run threads together, measure instructions, measure cycles, Run each thread in isolation, measure instructions, cycles 3. Compute WS = Σ (IPC SMT /IPC single )

16. SyNAR: Systems Networking and Architecture Group Estimating WS: Part II Run threads together, read hardware counters 4. Measure online hardware metrics AllConf Dcache FQ FP etc. 5. Correlate WS to each metric WS 1 AllConf 1 WS 2 AllConf 2 WS 3 AllConf 3 WS 4 AllConf 4... 6. Metric with highest correlation is the best predictor WS AllConf

17. SyNAR: Systems Networking and Architecture Group Result of the Model We know which metric best predicts symbiosis (WS) – IPC – Dcache – FQ – Composite – Score Measure Score online. If Score is high, there is high symbiosis.

18. SyNAR: Systems Networking and Architecture Group Scheduler Sample – Run many different co-schedules – Measure hardware counters Optimize – Predict which co-schedules have high symbiosis: those with high Score Schedule – Select co-schedules that are predicted symbiotic (with high Score )

19. SyNAR: Systems Networking and Architecture Group Performance Results

20. SyNAR: Systems Networking and Architecture Group Summary New processor motivated a new problem: resource contention Addressed by co-scheduling symbiotic threads Challenge: which threads are symbiotic? Solution: heuristic based on hardware counters On average 9% speedup