© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 1 Administrative Issues  Exam date candidates  CW 7 * Feb 14th (Tue): * Feb 16th (Thu):  CW 8 * Feb 24th (Fri):  CW 9 * Feb 28th (Tue): * Feb 29th (Wed): (cool date for an exam ) * March 1st (Thu): 10-12

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 2 Multiprocessing Support Tannenbaum (3. ed) Chapter 8 Parallel computer architectures OS design issues

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 3 Parallel computer architectures  Is parallel computing a good idea?  Which parallel computing designs exist?

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 4 Parallel Processing Motivation P1 P3 P2Input V O/P  Single Processor is a Single Point Of Failure (SPOF)  Not tolerable in critical applications (e.g. crash while writing thesis  )  Redundancy helps, e.g. Triple Modular Redundancy (TMR) Why not just double? CPUs usually have other detection mechanisms for critical failures…

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 5 Parallel Processing Motivation Performance gain through clock rate Power consumption? Implications?

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 6 Parallel Processing Overview  Flynn’s Taxonomy  Single instruction single data (SISD) stream no parallel operation  Single instruction multiple data (SIMD) stream vector and array processors, MMX & SSE instructions  Multiple instruction single data (MISD) stream never implemented  Multiple instruction multiple data (MIMD) stream multicore, multiprocessor

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 7 Parallel Processing Overview  Past: Multithreaded Processors  No real parallelism, just increased thread switching  Choose two threads  If one blocks, the other one runs after only  Present: Symmetric Multi-Processors SMPs  Uniform Memory Access (UMA)  Nonuniform Memory Access (NUMA)  Future: Heterogeneous MP  General purpose CPUs  General purpose GPUs  Specific coprocessors  Specific accelerators  Reconfigurable HW

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 8 SMPs Shared-memory multiprocessor Message-passing multicomputer Distributed System

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 9 SMPs: Bus-based UMA No caching Caching Caching + private memory Scalability?

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 10 SMPs: Crossbar-switched UMA Scalability?

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 11 SMPs: Multistage Network UMA Network of 2x2 switches A  X,X  A A  Y,Y  A B  X,X  B B  Y,Y  B Memory addressing scheme

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 12 SMPs: Multistage Network UMA Scalability? n CPUs x n memory units: n/2(log 2 n) switches Other problems? 011 sends READ (a) to module sends READ (b) to module 001

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 13 Intel’s Single-Chip Cloud Computer (SCC)

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 14 SMPs: Non-Uniform Memory Access (NUMA)  Single address space  Load/Store interface  Access times differ!

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 15 Parallel computer architectures  Is parallel computing a good idea?  At least there is no better idea right now...  Clock rate improvement is too expensive (heat dissemination)  Which parallel computing designs exist?  Flynn‘s taxonomy: SISD, SIMD, MISD, MIMD  MIMD: Multithreaded, SMPs, Heterogeneous Multiprocessors  SMPs: UMA, NUMA

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 16 Multiprocessor OS Types  Private OS per node  Master/Slave  Shared OS

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 17 Private OS per node  Shared HW, CPUs & memory partitioned statically  Soft Resources? Processes, pages, disk…

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 18 Master/Slave Central coordinating instance Pros?Cons?

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 19 Shared OS Now every user process can invoke the OS locally  No master CPU bottleneck, yet resource sharing  Other problems?

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 20 Multiprocessor OS issues  Synchronization  Switching off interrupts?  TSL?  Covered in the DS part  Scheduling  What to run?  Where to run?  Which combinations?  Covered in the exercise

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 21 Multiprocessor Synchronization Can TSL do the job?

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 22 Multiprocessor Synchronization

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 23 Multiprocessor Scheduling: Time Sharing

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 24 Multiprocessor Scheduling: Space Sharing

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 25 Multiprocessor Scheduling: Gang Scheduling

© DEEDS – OS Course WS11/12 Lecture 10 - Multiprocessing Support 26 Multiprocessor Scheduling: Gang Scheduling