1 MIMD Computers Based on Textbook – Chapter 4

2 PMS Notation (pg. 59) (Bell & Newell, 1987) Similar to a block notation, except using single letters Can augment letter with ( ) containing attributes

3 PMS Notation (Table 3.1) PProcessor (decoding & execution) MMemory (registers, cache, main, secondary) SSwitch (simple or complex) LLink: (often a line, often omitted) TTransducer (I/O device changing representation) KController (generates microsteps for single operations applied externally) DData processing (usually arithmetic, etc.) CComputer ( P, M, others – complete system)

4 MIMD – Multiple data stream Multiple control (instruction) streams Generally considered asynchronous Multiprocessor: single integrated system containing multiple PCs, each capable of executing an independent stream of instructions, but is an integrated system for moving data among PC, memory, I/O MIMD - Definition

5  Can use each PC for a different job – multiprogramming  Our interest: use for one job MIMD – General Usage

6 Granularity/Coupling Course Grain/Loosely Coupled: infrequent data communication separated by long periods of independent computations Fine Grain/Tightly Coupled: frequent data communications, usually in small amounts Grain: determined by program subroutines, basic blocks, stream or machine level

7 Characterized by how data/information from one PC is made available to other PCs Shared memory Message passing  Fixed connection  Distributed memory Types of MIMD

8 In distributed memory –> interconnection n.w. Features: Bandwidth: bytes per second Bisection bandwidth Latency: total time from transmission to reception Concurrency: number of independent connections that can be made Switches

9 Hybrid Computers – Mixed Type Saw many in the class presentations Many shared memory PC have some local memory NUMA: non-uniform memory access – some memory locations have larger access time Clusters: group of shared memory PC’s plus memory “separated” from other clusters; clusters message passing between clusters

10  Interprocessor communication via R/W instructions  Memory: maybe physically distributed (banks), may have different access times, may collide in switch  Memory latency maybe “long”, variable  “Messages” thru switch generally one “word”  Randomization of request maybe used to reduce memory collisions Shared Memory Features

11 Message Passing Features Aka Distributed Memory  Interprocessor communication via send/receive instructions  R/W refer to local memory  Data maybe collected into long messages before sending  Long transmissions may mask latency  Global scheduling maybe used to avoid message collisions

12 Scale: number link/PC - constant Ring Topology (Fig. 4.4)

13 Scale: number link/PC – constant (unless?) Mesh Topology - Torus (Fig. 4.5)

14 Scale: number link/PC –increases logarithmically Hypercube (Fig. 4.6)

15 L – Link (often omitted) Direction: Unidirectional or bidirectional Bandwidth (B): In bytes/sec Latency (R): Start of send to delivery of first byte Single PC to PC or pipelined (each link can be occupied) Specification of Network