1. M U N - February 17, 2005 - Phil Bording1 Computer Engineering of Wave Machines for Seismic Modeling and Seismic Migration R. Phillip Bording February 17, 2004 0 Max Address Husky Energy Chair in Oil and Gas Research Memorial University of Newfoundland

2. Session 1 History of Design Tyco Brahe Napier Charles Babbage – mechanical design John Atanasoff – Storage – spinning capacitor - Konrad Zuse - Floating Point Mauchley and Ekert von-Neumann Harvard memory – code memory - data Princeton memory code and data

3. Session 2 Current Design Issues Scaling laws Moore’s Law Transistors – VLSI Memory – Technology Division of Design The memory Challenge The processor Challenge The ILLIAC – PEPE IBM 7094 IBM 360/44 IBM 360/95 Array Processors the software of array processor calls Programming Models vectors shared memory distributed memory

4. M U N - February 17, 2005 - Phil Bording4 Lamda Rules

5. M U N - February 17, 2005 - Phil Bording5 Division of design ALU Memory ALU One Company Company B Company A Weak Link

6. M U N - February 17, 2005 - Phil Bording6 Moore’s Laws Every 18 months the density of transistors on a VLSI chip doubles The investments of $ doubles with every new VLSI plant

7. M U N - February 17, 2005 - Phil Bording7 Illiac 8 X 8 Processors Nearest Neighbor Connections

8. M U N - February 17, 2005 - Phil Bording8 Parallel Ensemble Processing Elements - PEPE P0 Pn-3Pn-2Pn-1Pn.. Data Inputs Radar Processing Computer Associative Computing Data Outputs

9. M U N - February 17, 2005 - Phil Bording9 IBM Machines Early 1960’s 7094, 36 bit arithmetic 1600 and 1400 processors completely different Middle 1960’s New Machine – IBM 360 36 bit words, but memory parity was added 8 bit byte + 1 bit parity Uniform business machine architectures 32 and 64 bit floating point Not any industry standard for format of floating point

10. M U N - February 17, 2005 - Phil Bording10 Array Processors IBM and CDC designed DMA processors – Direct Memory Access Frees the main processor to compute Allows separate simple processors to do the i/o The idea translated into attached processors for arithmetic processing

11. M U N - February 17, 2005 - Phil Bording11 Array Processors Arrays of data are moved to a local very high speed memory – fast registers Arithmetic is performed by special instructions passed to array processor CPU Array Processor

12. M U N - February 17, 2005 - Phil Bording12 Software Design Issues Vector Programming Cache Programming Message Passing Programming NUMA Programming Grid Programming ALL of these memory operations have a Fixed Cost Code Performance Improvements are dominated by fixed costs

13. M U N - February 17, 2005 - Phil Bording13 Hardware Design Issues 10 Years equals 100 Fold Speedup Memory Latency – cost of getting the first word is a constant Wires have failed to scale Bigger cache memories are slower Code Performance Improvements are dominated by fixed costs

14. M U N - February 17, 2005 - Phil Bording14 Linear Address Space Address Pointer 0 Max Address Latency is the time to access the first word Bandwidth is the rate of accessing successive words

15. M U N - February 17, 2005 - Phil Bording15 von Neumann Architecture Princeton Address Pointer Arithmetic Logic Unit (ALU) Memory Program Counter Pc = Pc + 1 Data/Instructions Featuring Deterministic Execution

16. M U N - February 17, 2005 - Phil Bording16 Cache Memory Architecture Address Pointer Memory Cache Memory CACHECONTROLCACHECONTROL Main Memory is large and slow. Cache is much smaller and much faster. Control logic control keeps the main memory coherent. Featuring Non-Deterministic Execution

17. M U N - February 17, 2005 - Phil Bording17 Cache Memory - Three Levels Architecture Address Pointer Memory Multi- Gigabytes Large and Slow 160 X 16X L3 Cache Memory Cache Control Logic L2 Cache Memory L1 Cache Memory 2X 8X 16 Megabytes 128 Kilobytes 32 Kilobytes 2 Gigahertz Clock Featuring Really Non-Deterministic Execution

18. M U N - February 17, 2005 - Phil Bording18 Programming Models for Parallel Computing

19. M U N - February 17, 2005 - Phil Bording19 Multiple Address Pointers Program Address Spaces 0 Max 0 Max Distributed Computing Message Passing Interface

20. M U N - February 17, 2005 - Phil Bording20 Distributed Computing with Message Passing Multiple Address Pointers Program Address Spaces Messages Left and Right

21. M U N - February 17, 2005 - Phil Bording21

22. M U N - February 17, 2005 - Phil Bording22 Multiple Address Pointers Global Program Address Space 0 n-1 n 2n-1 2n 3n-1 3n 4n-1 Local Local Address and Cache Bus with Conflict Resolution Multi-Threading OpenMP Programming Model

23. M U N - February 17, 2005 - Phil Bording23 Uniqueness of Store Multi-Threading Multiple Address Pointers Program Address Space Duplicate Pointers to the same Location – Conflict on storing a result So who is managing the multiple pointers? It is the programmers responsibility. 0

24. M U N - February 17, 2005 - Phil Bording24 Multiple Bank Memory Systems Starting + 1 +2 +3 Address +N +2N +3N Mod 4 Memory Banks Bank 0 1 2 3 Vector Programming Model