1 MIMD Computers Module 4

2 PMS Notation (Bell & Newell, 1987) Similar to a block notation, except using single letters Can augment letter with ( ) containing attributes

3 PMS Notation PProcessor (decoding & execution) MMemory (registers, cache, main, secondary) SSwitch (simple or complex) LLink: (often a line, often omitted) TTransducer (I/O device changing representation) KController (generates microsteps for single operations applied externally) DData processing (usually arithmetic, etc.) CComputer ( P, M, others – complete system)

4 MIMD – Multiple data stream Multiple control (instruction) streams Generally considered asynchronous Multiprocessor: single integrated system containing multiple PCs, each capable of executing an independent stream of instructions, but is an integrated system for moving data among PC, memory, I/O MIMD - Definition

5  Can use each PC for a different job – multiprogramming  Our interest: use for one job MIMD – General Usage

6 Granularity/Coupling Course Grain/Loosely Coupled: infrequent data communication separated by long periods of independent computations Fine Grain/Tightly Coupled: frequent data communications, usually in small amounts Grain: determined by program subroutines, basic blocks, stream or machine level

7 Characterized by how data/information from one PC is made available to other PCs Shared memory Message passing  Fixed connection  Distributed memory Types of MIMD

8 Shared memory and message-passing multiprocessors Types of MIMD

9 In distributed memory –> interconnection n.w. Features: Bandwidth: bytes per second Bisection bandwidth (the bps across an interface that partitions the network into two equal groups.) Latency: total time from transmission to reception Concurrency: number of independent connections that can be made (bus has concurrency = 1) Switches

10 Many shared memory PC have some local memory NUMA: non-uniform memory access – some memory locations have larger access time Hybrid Computers – Mixed Type Shared memory multiprocessor with private memories.

11 Clusters: group of shared memory PC’s plus memory “separated” from other clusters; clusters message passing between clusters Hybrid Computers – Mixed Type

12  Interprocessor communication via R/W instructions  Memory: maybe physically distributed (banks), may have different access times, may collide in switch  Memory latency maybe “long”, variable  “Messages” thru switch generally one “word”  Randomization of request maybe used to reduce memory collisions Shared Memory Features

13 Message Passing Features Aka Distributed Memory  Interprocessor communication via send/receive instructions  R/W refer to local memory  Data maybe collected into long messages before sending  Long transmissions may mask latency  Global scheduling maybe used to avoid message collisions

14 Scale: number links/PC - constant Ring Topology Message passing multiprocessors distinguished from one another by their topology

15 Scale: number link/PC – constant (unless?) Mesh Topology - Torus

16 Scale: number link/PC –increases logarithmically Hypercube

17 L – Link (often omitted) Direction: Unidirectional or bidirectional Bandwidth (B): In bytes/sec Latency (R): Start of send to delivery of first byte Single PC to PC or pipelined (each link can be occupied) Specification of Network