IPDPS Looking Back Panel Uzi Vishkin, University of Maryland

Moderator: What has gone well? IPDPS a big success. Has become top facilitator for: Specific technical contributions, and Open debate of challenges – e.g., this panel Warm congrats to Viktor and all others! 2

What also has gone well Parallel PRAM algorithmic theory, second in magnitude only to the serial algorithmic theory Won the “battle of ideas” in the 1980s. Repeatedly: Challenged without success  no real alternative! 3

Dream opportunity Limited interest in parallel computing evolved into quest for general-purpose parallel computing in mainstream machines Example many-core desktops So far for the good news Are we doing everything we can to ensure that many- cores are not rejected by programmers? Recall Einstein’s observation: A perfection of means, and confusion of aims, seems to be our main problem What should be our aims? 4

“If you find yourself in a hole, stop digging” Moderator: What has gone wrong We found ourselves in a hole: most programmers can’t handle today’s (multicore) desktops Moderator: What was surprising We keep digging Why are we in trouble 1940’s Stored-program & program-counter  For serial comp: knowledge of algorithms was low priority for arch (& knowledge of arch was low priority for alg people) No such agreed “bridge” for many-cores. Still: -Industry realizes the need to reinvent computing for parallelism but is stuck with short-term pressures/culture. Academia hold on. -Education Architects pay too limited attention to (this time parallel) algorithms. How will they know to build machines that are easy to program? Instead, funding guides using problematic designs 5

What follows -Are many -core architectures doomed to mismatch parallel algorithms and ease-of-programming (EoP)? -What difference can a matching arch make? -What is feasible? 6

How come that most programmers can’t handle today’s (multicore) desktops? Hypothesis: Flawed architecture foundation Origin: ‘build-first figure-out-how-to-program-later’ Parallel languages: fitted flawed architectures then standardized Who can save the field and promote the aim of ease-of-programming (EoP)? Industry (perfecting means) -Follow-up architectures fit language standards  remain flawed -Insufficient competition Academia (perfecting means) -Consider a vendor-backed flawed system. Wonderful opportunity for our originality-seeking publications culture: * The simplest problem requires creativity  More papers * Cite one another if on similar systems  high # citations coupled with ‘industry impact’ - Ultimate job security – By the time the ink dries on these papers, next flawed ‘modern’ ‘state-of-the-art’ system. Culture of short-term impact 7

Anecdotal Validation (?) Breadth-first-search (BFS) example 42 students: joint UIUC/UMD course -<1X speedups using OpenMP on 8-processor SMP -7x-25x speedups on 64-processor XMT FPGA prototype [Built at UMD] What’s the big deal of 64 processors beating 8? Silicon area of 64 XMT processors ~= 1-2 SMP processors Questionnaire Rank approaches for achieving (hard) speedups: All students, but one : XMTC ahead of OpenMP Order-of-magnitude advantage on teachability (MS, HS & up, SIGCSE’10) SPAA’11: >100X speedup on max-flow relative to 2.5X on GPU (IPDPS’10) Fleck/Kuhn: research too esoteric to be reliable  exoteric validation! What has gone wrong Only heroic programmers can exploit the vast parallelism in current machines – The Future of Computing Performance: Game Over or Next Level?, Report by NAE Conclusion Fund power.. Reward alert: Try to publish a paper boasting easy to obtain results  EoP: 1. Badly needed. Yet, 2. A lose-lose proposition. 8

Parallel Random-Access Machine/Model PRAM: n synchronous processors all having unit time access to a shared memory. Reactions You got to be kidding, this is way: - Too easy - Too difficult: Why even mention processors? What to do with n processors? How to allocate processors to instructions?

Immediate Concurrent Execution 10 ‘Work-Depth framework’ SV82, Adopted in Par Alg texts [J92,KKT01]. ICE basis for architecture specs: V, Using simple abstraction to reinvent computing for parallelism, CACM 1/2011 Similar to role of stored-program & program-counter in arch specs for serial comp

Algorithms-aware many-core is feasible Algorithms Programming Programmer’s workflow Rudimentary yet stable compiler PRAM-On-Chip HW Prototypes 64-core, 75MHz FPGA of XMT [ SPAA98..CF08] Toolchain Compiler + simulator HIPS’ core interconnection network IBM 90nm: 9mmX5mm, MHz [HotI07] FPGA design  ASIC IBM 90nm: 10mmX10mm 150 MHz Architecture scales to cores on-chip XMT homepage: or search: ‘XMT’

Where are your specs? What is your par alg abstraction? ‘ First-specs then-build’ is “not uncommon”.. for engineering I see only 2 options for architects: A.1. Go through parallel algorithms immersion 2. Develop abstraction that meets EoP 3. Develop specs 4. Build B.Start from abstraction with proven EoP

Sociologists of science Debates between adherents of different thought styles consist almost entirely of misunderstandings. Members of both parties are talking of different things (though they are usually under an illusion that they are talking about the same thing). They are applying different methods and criteria of correctness (although they are usually under an illusion that their arguments are universally valid and if their opponents do not want to accept them, then they are either stupid or malicious) 13