1. شبکه های میان ارتباطی 1 به نام خدا دکتر محمد کاظم اکبری مرتضی سرگلزایی جوان http://crc.aut.ac.ir

2. Taxonomy 2

3. MIMD Multiprocessor (shared memory) IN P1 P2 Pn M1 M2Mn Processors Interconnection Network Memory modules (Tightly Coupled Architecture) 3

4. Shared Memory 4 Uniform Memory Access (UMA) Tightly Coupled system Non-Uniform Memory Access (NUMA) Loosely Coupled system Cedar from University of Illinois BBN Butterfly Cache Only Memory Access (COMA) Using global distributed caches Kendal Square Research-1 (KSR-1) 4

5. MIMD (cont.) Global Interconnection Network (Global IN) Global Memory GM 1 Global Memory GM 2 Global Memory GM n P1P1 P2P2 PnPn CINCIN CM 1 CM 2 CM 3 P1P1 P2P2 PnPn CINCIN CM 1 CM 2 CM 3 P1P1 P2P2 PnPn CINCIN CM 1 CM 2 CM 3 (Loosely Coupled Architecture) - Cedar 5

6. MIMD (cont.) P1P1 M1M1 P2P2 M2M2 PnPn MnMn Interconnection Network (IN) (Loosely Coupled Architecture) – BBN Butterfly 6

7. MIMD (cont.) Interconnection Network (IN) D1D1 C1C1 P1P1 D2D2 C2C2 P2P2 DnDn CnCn PnPn (COMA Architecture) 7

8. MIMD (cont.) Multicomputer (Message passing) IN P1 M1 P2 M2 Pn Mn 8

9. MIMD (cont.) Data flow machine an instruction is ready for execution when data for its operands have been made available Purely self-contained No program counter 9

10. SIMD Array Processor centralized control unit

11. MISD Pipelined vector processor

12. MISD (cont.) Systolic array 12

13. Hybrid Architecture Combine features of different architectures to provide better performance for parallel computations. Two type of parallelism Control parallelism (MIMD) Data parallelism (SIMD) 13

14. Special Purpose Devices Artificial Neural Networks (ANN) Fuzzy logic 14

15. Neural Networks (Definition) A large number of PEs Connected in Parallel Capable of learning Adaptive to changing Able to cope with serious disruptions Power of Connectivity Power of Processors vs 15

16. Fuzzy logic (Definition) Approximate reasoning Formal principals of reasoning 16

17. Interconnection Network (IN) The measure of an IN is “how quickly it can deliver how much of what’s needed to the right place, reliably and at good cost and value”. 17

18. Performance Criteria for IN Latency Transit time for a single msg. Bandwidth how much msg. traffic the IN can handle, e.g., Mbytes/s Connectivity How many immediate neighbors each node has, and how often each neighbor can be reached Hardware cost What fraction of the total hardware cost the IN represents E.g., wires, switches, connectors, arbitration logic, … 18

19. Performance Criteria for IN (cont.) Reliability Redundancy paths, Functionality Additional functions performed by the IN, such as combining of msg. and fault tolerance e.g., data routing, interrupt handling, request/ message combining, coherence Scalability The ability to be expandable 19

20. Definitions Node degree: node degree is the number of links (edges) connected to the node Diameter: the diameter of a network is defined as the largest minimum distance between any pair of nodes. The minimum distance between a pair of nodes is the minimum number of communication links (hops) that data from one of the nodes must traverse in order to reach the other node. Network Size The number of nodes in the IN 20

21. Data Routing Functions in data routing Shifting Rotation Permutation (one-to-one) Broadcast (one-to-all) Multicast (many-to-many) Personalized communication (one-to-many) Shuffle / Exchange 21

22. Types of IN Static Networks Dynamic Networks 22

23. Static Networks Shared Bus Degree = 1 Diameter = 1 23

24. Static Networks (cont.) Linear Array Degree = 2 Diameter = n-1 24

25. Static Networks (cont.) Ring Degree = 2 Diameter: unidirectional: n-1 bidirectional: Ceil(n-1)/2 25

26. Static Networks (cont.) Binary tree Degree: Leaf=1 Root=2 Others=3 Diameter: 2(h-1) 26

27. Static Networks (cont.) Fat tree. Degree and Diameter is the same as binary tree Due to heavy traffic towards root, the number of links gradually increases (e.g., CM-5). 27

28. Static Networks (cont.) 28 Star. Degree: Central = n-1 Others = 1 Diameter= 2

29. Static Networks (cont.) Shuffle(s n-1 s n-2... s 0 ) = s n-2 s n-3... s 0 s n-1 Exchange(s n-1 s n-2... s 1 s 0 ) = s n-1 s n-2... s 1 s 0 SourceDestination 000  000 001  010 010  100 111  111 100  001 101  011 110  101 011  110 29

30. Shuffle-Exchange Network For N=8 Applications: The shuffle-exchange network provides suitable interconnection patterns for implementing certain parallel algorithms, such as polynomial evaluation, Fast Fourier Transform (FFT), sorting, and matrix transposition. 30

31. Static Networks (cont.) Mesh. Degree: Corner= 2 Sides = 3 Middle= 4 Diameter= 2(n-1) 31

32. Mesh Routing Algorithm Simple routing algorithm routes a packet from source S to destination D in a mesh with n 2 nodes. 1. Compute the row distance R as 2. Compute the column distance C as 3. Add the values R and C to the packet header at the source node. 4. Starting from the source, send the packet for R rows and then for C columns. 32

33. Example (Mesh) 33 to route a packet from node 6 (i.e., S=6) to node 12 (i.e., D =12), the packet goes through two paths, as shown in the figure:

34. Static Networks (cont.) Illiac Degree= 4 Diameter= n-1 chordal ring 34

35. Static Networks (cont.) Torus Degree= 4 Diameter= 2(Ceil(n/2)) 35

36. HyperCube Degree= n Diameter= n Address Bits= n Dimensions= n Neighbors= n 36 Static Networks (cont.)

37. Example Embedding a 4-by-4 mesh in a 4-cube 37

38. Static Networks (cont.) n-Mesh Degree: Corner= n Internal= 2n n < Others < 2n Diameter= 38

39. Static Networks (cont.) k-Ary n-cube Degree: If k=2 then Degree = n If k>2 then Degree = 2n Diameter= (a) 4-ary 2-cube network (b) 3-ary 3-cube network 39

40. Cache Coherence Multiprocessor environment Cache dedicated to each processor Cache coherence problem How to keep multiple copies of the data consistent during execution ? 40

41. Cache Coherence Mechanisms 1. Hardware-based schemes Snoopy cache protocols If INs have broadcast features Directory cache protocols No broadcast features in INs 2. Software-based schemes 3. Combination 41

42. Cache Coherence Mechanisms (cont.) Action taken on Read Miss Write Hit Write Miss 42

43. Snoopy Cache Protocol 43 A two-processor configuration with copies of data block x  write-through  write-back

44. Centralized Directory Protocols Full-map protocol directory 44

45. Scalable Cache Coherency 45

46. Classification of Dynamic Networks 46

47. Dynamic Networks (Crossbar) 47

48. Dynamic Networks (Single-Stage) In Single-Stage Network any permutation can be reached by at most 3(log N 2 ) -1 pass. 48

49. Multi Stages - Blocking Example: Multi Stage Cube, Omega 49

50. Multi Stages – Nonblocking Example: Three-stage Clos 50

51. Dynamic Networks (Clos) 51

52. Multi Stages - Rearrangeable Example: 8-to-8 (Benes) 52

53. Interconnection Design Decisions Considerations about selecting the Architecture of Interconnection Network Operation Mode Control Strategy Network Topology Switching Methodology Functional characteristics of the switch 53

54. Interconnection Design Decisions Operation mode: Synchronous Asynchronous Combined Control Strategy Centralized control Distributed control Switching methodology circuit switching packet switching integrated switching 54

55. ابر و باران 55 http://crc.aut.ac.ir