New trends in parallel computing Go, Julia, Go! New trends in parallel computing

Covered topics Modern parallel languages for shared and distributed memory computing golang Julia

Motivation The search for elegant and high—performance parallel language is neverending C-like leads in performance Python leads in comprehension and ease of use Matlab is the favourite of engineers Some people like it functional: Erlang So what has been going on recently?

Go Also known as ‘golang’ From big influencers in the field compiled, statically typed, object oriented, garbage-collected built-in concurrency primitives and CSP-like parallelism From big influencers in the field (Ken) Thompson, (rob) Pike, Cox, Griemser Backed by a big company Guess which one

Go Communicating sequential processes (Hoare) “Do not communicate by sharing memory; instead, share memory by communicating.” Pattern of interaction in concurrent systems Formally: process algebra, process calculus Implements IPC via message passing (even in shared memory) through shared channels Helpful for development of verified concurrent systems Handy for handling critical sections, pooled operations

Go Key properties Statically typed and well-scalable (like C++, Java) Light (like python and ruby) Far away from the unnecessary complexity of C++-like langs Large built-in library Fast compiler (statically linked binaries) Testing framework

Go Goroutines A goroutine is a lightweight thread of execution. Implemented as a function or anonymous function. Semi-cooperatively scheduled

Go Channels Channels are the pipes that connect concurrent goroutines You can send values into channels from one goroutine and receive those values into another goroutine Typed by values they convey Send a value into a channel using the channel <- syntax. Here we send "ping" to the messages channel we made above, from a new goroutine.

Go Channels can be used for synchronization

Go Channels can be used for synchronization directed (good for type safety)

Go Channels can be used for synchronization directed (good for type safety) part of a select statement

Go Channels can be used for synchronization directed (good for type safety) part of a select statement with a timeout

Go Channels can be used for synchronization directed (good for type safety) part of a select statement with a timeout or with a default value (non-blocking)

Julia High-level, high-performance dynamic language for technical computing Distributed parallel execution, plentiful libraries LLVM-based JIT compiler ‘One-sided’ implementation of message passing programmer needs to explicitly manage only one process in a two-process operation Communication resembles higher-level operations like calls to user functions

Julia two key primitives: remote references and remote calls an object that can be used from any process to refer to an object stored on a particular process two flavors - Future and RemoteChannel Remote call request by one process to call a certain function on certain arguments on another (possibly the same) process returns a Future to its result

Julia Other useful concepts Synchronization Scheduling Distributed garbage collection Channels Shared arrays