Parallel Programming in Contemporary Programming Languages James Hoekzema

Why The optimal environment is not always available. Code needs to run on machines with different specs Operating System does not support certain libraries The environment is a restricted Virtual Machine Licensing for algorithms is not always cheap High end components are expensive “Fast Enough” is better than nothing at all

What JavaScript - Using parallel programming and concepts in the browser Go - Built-in features for parallelism beside and to scale a server LabVIEW - Graphical programming for smart compiler parallelism

JavaScript (in the Browser)

Why Networking is slow and unpredictable. Different providers, different regions, different services levels Connection is outside of your control Scaling servers is costly. On a server, more people connecting means slower service all around The user is providing computation power to you, use it! Much wider support than plugins like Java or Flash. Plugins require downloading, updating, loading up different virtual machines Browsers are less prone to versioning that affects performance

What Asynchronous Single-thread JS only runs when needed, so networking is nonblocking Tasks are put into a queue for execution during “downtime” Web Workers Simplified JS environment on another thread, uses message passing interface Can be bound to a webpage or to a domain (Shared Worker) Data is deep copied unless a buffer is specified which then transfers control WebGL Essentially OpenGL for a web page, runs on the Canvas Element Allows access to GPU computing (or at least accelerated Graphics Computing)

Example (Asynchronous) var fact = []; function ajax(method, url, callback) { //create request var xhr = new XMLHttpRequest(); //set callback function xhr.onload = function(){ callback(JSON.parse(xhr.responseText)); }; //specify request xhr.open(method, url); //send request xhr.send(); } function factorial(n) { if(n > 1){ return n * factorial(n - 1); }else{ return 1; ajax('GET', 'primes.json', function(primes){ //executes after factorial var result = 0; var len = Math.min(primes.length, fact.length); for(var i = 0; i < len; i++){ result += primes[i] / fact[i]; } console.log(result); }); //executes after request, before response for(var i = 0; i < 20; i++){ fact.push(factorial(i + 1)); primes: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, ...] fact: [1, 2, 6, 24, 120, 720, 5040, 40320, ...] ------------------------------------------------- result: 4.73863870268622

Example (Web Workers) Polynate var worker1 = new Worker('worker.js'); worker1.postMessage({ type: 'sum', data: [2, 3, 5, 7, 11, 13, 17, 19, 23] }); worker2.postMessage({ data: [29, 31, 37, 41, 43, 47, 53, 59] var sum = 0, count = 0; worker1.onmessage = worker2.onmessage = function(event) { count++; sum += event.data; if(count == 2){ console.log(sum); } //worker.js onmessage = function(event) { if(event.data.type == 'sum'){ postMessage(sum(event.data.data)); } //other parallel methods function sum(arr) { var len = arr.length, sum = 0; for(var i = 0; i < arr.length; i++){ sum += arr[i]; return sum; result: 440 Polynate

Example (WebGL)

Go (Golang)

Why Golang is built for Concurrency (and thus parallelism) Simply prefixing a function call with “go” spins up a new goroutine You can compile for different number of processors Many of the standard library features already support concurrency Golang’s Standard Library is well-suited for servers A simple server can be written in less than 10 lines of code Go makes JSON easily translatable into and out of data structures Go provides an intuitive templating engine

What “go <function call>” This call spins up a goroutine that can run on any processor or concurrently Not the same as MPI as it is not made for distributed computing Channels Channels allow you to pass data around between functions and routines Channels can have multiple access points, so you can have several routines “listening” on a channel and when data comes through, one will grab it and switch to being busy

Channels Routine 3 Routine 3 Routine 3 Routine 3 Main Routine 2 2. Main makes routines and passes them both channels 6. Routine 3 claims third piece of data, no longer listening 1. Main makes 2 channels, data and result Routine 3 Routine 3 Routine 3 Routine 3 data Main Routine 2 Routine 2 Routine 2 Routine 2 3. Main sends data over channel and listens to result channel Routine 1 Routine 1 Routine 1 Routine 1 4. Routine 1 claims first piece of data, no longer listening 5. Routine 2 claims second piece of data, no longer listening 8. Main receives and processes the results 7. The routines finish and send their results back on the result channel result

Example Code 1 4 27 package main import "fmt" func main() { data := make(chan int, 3) result := make(chan int, 3) go nToTheN(data, result) for n := 1; n < 4; n++ { data <- n } fmt.Println(<-result) func nToTheN(data chan int, result chan int) { n := <-data res := 1 for i := 0; i < n; i++ { res *= n } result <- res 1 4 27

LabVIEW

Why LabVIEW automatically parallelizes your code If you put two loops next to each other, LabVIEW runs them in parallel You can force parallelism in some places like for loops LabVIEW is built for instrumentation (particularly signal processing) You can read signals and process them at the same time without having to worry about delays or order LabVIEW comes with a graphical interface builder By default, every subVI has a “Front Panel” that shows controls and views

What Graph/Flowchart Based Programming Graphical programming where subVIs are connected by wires Independent parts of the graph/flowchart can thus be run simultaneously Physical Hardware Integration LabVIEW provides FPGA, GPU (CUDA), etc. subVIs to use in your code You can literally “draw” the structure, like a systolic array (grid)

Front Panel

Examples (Basic Parallelism) Execution of independent tasks Execution of independent branches Execution of parallel data operations

Example (Parallel loops and FPGA)

Example (Pipelining)

Questions? What languages would you like to see next time? Mobile/Tablet? Cloud? GPU? Go deeper on these 3?