Conclusions on CS3014 David Gregg Department of Computer Science University of Dublin, Trinity College

Interesting Times In the mid 2000’s power became the main limiter of processor performance No longer possible to keep scaling clock speed Performance must come from other sources After decades of rejecting parallel computing as too hard for most purposes the industry switched to multicore almost overnight in 2005

Moore’s Law Moore’s law is still alive and well At least for the moment Feature densities will continue to double around every 18-24 months More sophisticated, faster cores will still arrive But the rate of improvement of single core has been much, much slower than between 1984 and 2005 Also have lots of parallelism on chip

Parallel Architectures Many parallel architectures have been proposed over the years Instruction-level parallel Vector Multi-threaded Shared-memory multiprocessor Distributed memory multiprocessor GPU, FPGA, hardware accelerators, etc.

Parallel Architectures Existing types of architecture, which were mostly originally designed for supercomputing, will be implemented on single chips The same ideas that worked in 1970’s supercomputers will work in 21st century single chip, and stacked chip processors

Software The big problem of parallel computing has always been software “Any steps that are programmed by the operator, who sets up the machine, should be set up only in a serial fashion. It has been shown over and over again that any departure from this procedure results in a system that is much too complicated to use.” J. P. Eckert 1946

Software The big unknown with parallel computing is how we will build software at an acceptable cost Large diversity of proposed new programming models and languages OpenMP, Intel Array Building Blocks, X10, stream processing languages, etc. Also domain-specific languages E.g. Halide for image processing

Software Over time the successful languages and programming models will emerge The most influential languages are hardly ever the most widely used ones Historical examples such as Lisp, Algol 60, Simula 67, Occam This will probably leave a large legacy of Programs in languages that are forgotten Single-threaded COBOL code

This time it’s different There are some differences this time Industry sees no alternative to parallelism We must build parallel software if we want improved performance For all kinds of applications Even if we don’t want to

But how different? Moore’s law continues to provide a doubling of transistors every 24 months or so But the number of cores tends not to double Companies like Intel still make lots of processors with 4 cores Some with 18 cores, hardly any with more Predictions of doubling of cores every two years have not happened

But how different? The big difference has been the move to mobile, battery-powered computing Demand for low power computing Lower energy leads to real improvements in battery life It’s not clear that people upgrade their desktop machine very often If they have one

More differences Hardware trade-offs are also different Relative costs of communication, shared memory, etc. very different on a single chip compared to multiple chips Memory locality is really important And almost entirely application dependent Energy is often a key limit And influenced by memory locality 3D stacking will produce some surprises Embedded computing ever more important

Cheap Science Fiction Never forget wisdom of cheap sci fi: “All this has happened before; all this will happen again.” The same parallel architectures ideas appear again and again We have been failing to build parallel software for decades