1. Parallel Programming in Contemporary Programming Languages
James Hoekzema

2. Why The optimal environment is not always available.
Code needs to run on machines with different specs
Operating System does not support certain libraries
The environment is a restricted Virtual Machine
Licensing for algorithms is not always cheap
High end components are expensive
“Fast Enough” is better than nothing at all

3. What
JavaScript - Using parallel programming and concepts in the browser
Go - Built-in features for parallelism beside and to scale a server
LabVIEW - Graphical programming for smart compiler parallelism

4. JavaScript (in the Browser)

5. Why Networking is slow and unpredictable.
Different providers, different regions, different services levels
Connection is outside of your control
Scaling servers is costly.
On a server, more people connecting means slower service all around
The user is providing computation power to you, use it!
Much wider support than plugins like Java or Flash.
Plugins require downloading, updating, loading up different virtual machines
Browsers are less prone to versioning that affects performance

6. What Asynchronous Single-thread
JS only runs when needed, so networking is nonblocking
Tasks are put into a queue for execution during “downtime”
Web Workers
Simplified JS environment on another thread, uses message passing interface
Can be bound to a webpage or to a domain (Shared Worker)
Data is deep copied unless a buffer is specified which then transfers control
WebGL
Essentially OpenGL for a web page, runs on the Canvas Element
Allows access to GPU computing (or at least accelerated Graphics Computing)

7. Example (Asynchronous)
var fact = [];
function ajax(method, url, callback) {
//create request
var xhr = new XMLHttpRequest();
//set callback function
xhr.onload = function(){
callback(JSON.parse(xhr.responseText)); };
//specify request
xhr.open(method, url);
//send request
xhr.send(); }
function factorial(n) {
if(n > 1){
return n * factorial(n - 1);
}else{
return 1;
ajax('GET', 'primes.json', function(primes){
//executes after factorial
var result = 0;
var len = Math.min(primes.length, fact.length);
for(var i = 0; i < len; i++){
result += primes[i] / fact[i]; }
console.log(result);
});
//executes after request, before response
for(var i = 0; i < 20; i++){
fact.push(factorial(i + 1));
primes: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, ...]
fact: [1, 2, 6, 24, 120, 720, 5040, 40320, ...]
result:

8. Example (Web Workers) Polynate var worker1 = new Worker('worker.js');
worker1.postMessage({
type: 'sum',
data: [2, 3, 5, 7, 11, 13, 17, 19, 23]
});
worker2.postMessage({
data: [29, 31, 37, 41, 43, 47, 53, 59]
var sum = 0, count = 0;
worker1.onmessage =
worker2.onmessage = function(event) {
count++;
sum += event.data;
if(count == 2){
console.log(sum); }
//worker.js
onmessage = function(event) {
if(event.data.type == 'sum'){
postMessage(sum(event.data.data)); }
//other parallel methods
function sum(arr) {
var len = arr.length, sum = 0;
for(var i = 0; i < arr.length; i++){
sum += arr[i];
return sum;
result: 440
Polynate

9. Example (WebGL)

10. Go (Golang)

11. Why Golang is built for Concurrency (and thus parallelism)
Simply prefixing a function call with “go” spins up a new goroutine
You can compile for different number of processors
Many of the standard library features already support concurrency
Golang’s Standard Library is well-suited for servers
A simple server can be written in less than 10 lines of code
Go makes JSON easily translatable into and out of data structures
Go provides an intuitive templating engine

12. What “go <function call>”
This call spins up a goroutine that can run on any processor or concurrently
Not the same as MPI as it is not made for distributed computing
Channels
Channels allow you to pass data around between functions and routines
Channels can have multiple access points, so you can have several routines “listening” on a channel and when data comes through, one will grab it and switch to being busy

13. Channels Routine 3 Routine 3 Routine 3 Routine 3 Main Routine 2
2. Main makes routines and passes them both channels
6. Routine 3 claims third piece of data, no longer listening
1. Main makes 2 channels, data and result
Routine 3
Routine 3
Routine 3
Routine 3
data
Main
Routine 2
Routine 2
Routine 2
Routine 2
3. Main sends data over channel and listens to result channel
Routine 1
Routine 1
Routine 1
Routine 1
4. Routine 1 claims first piece of data, no longer listening
5. Routine 2 claims second piece of data, no longer listening
8. Main receives and processes the results
7. The routines finish and send their results back on the result channel
result

14. Example Code 1 4 27 package main import "fmt" func main() {
data := make(chan int, 3)
result := make(chan int, 3)
go nToTheN(data, result)
for n := 1; n < 4; n++ {
data <- n }
fmt.Println(<-result)
func nToTheN(data chan int, result chan int) {
n := <-data
res := 1
for i := 0; i < n; i++ {
res *= n }
result <- res 1 4 27

15. LabVIEW

16. Why LabVIEW automatically parallelizes your code
If you put two loops next to each other, LabVIEW runs them in parallel
You can force parallelism in some places like for loops
LabVIEW is built for instrumentation (particularly signal processing)
You can read signals and process them at the same time without having to worry about delays or order
LabVIEW comes with a graphical interface builder
By default, every subVI has a “Front Panel” that shows controls and views

17. What Graph/Flowchart Based Programming
Graphical programming where subVIs are connected by wires
Independent parts of the graph/flowchart can thus be run simultaneously
Physical Hardware Integration
LabVIEW provides FPGA, GPU (CUDA), etc. subVIs to use in your code
You can literally “draw” the structure, like a systolic array (grid)

18. Front Panel

19. Examples (Basic Parallelism)
Execution of independent tasks
Execution of independent branches
Execution of parallel data operations

20. Example (Parallel loops and FPGA)

21. Example (Pipelining)

22. Questions? What languages would you like to see next time?
Mobile/Tablet?
Cloud?
GPU?
Go deeper on these 3?