1. Processing Framework Sytse van Geldermalsen
Masters Grid Computing – University of Amsterdam
Internship at Amsterdam Medical Centre
Hello and welcome to my presentation.
I study grid computing at the university of Amsterdam and currently doing my internship here.
The context of my project I’m working on here is high performance computing on a single computer.
Especially here at the AMC with all the medical imaging processing, post processing of large amounts of data of patients in follow up projects/research, there is need of fast processing, as a large part of our group here is working with the grid.
My audience is a little divided, I some of you are more involved with the grid and high performance programming. I will try to stay basic and general, but nearing the end of the presentation it will become a little bit more technical.

2. Contents OpenCL Concepts Problems Research and projects
Processing Framework
Example
These are the contents of the presentation,
I will start with the library I am using to achieve high performance computing.
This is called OpenCL – or Open Computing Language and I’m going to talk about the key concepts behind it and why it makes it an attractive computing platform
There are some Concepts in the world of OpenCL that I will explain as they because I am going to use them throughout the presentation
OpenCL is a relatively new development (past 3 years) and there are some problems facing developers who write OpenCL programs, one of which is how fast and easy it is to write OpenCL programs
These problems are being tackled in various ways in current research to speed up application development.
With a lot of ideas presented in this research I aim to bring some of them together and introduce a new Processing Framework that will help developers program with OpenCL
I would also like to show a small example of what it would look like to conceptually write code to use this framework.

3. OpenCL Requires vendor support Portable
ARM, AMD, Intel, Apple, Vivante Corporation, STMicroelectronics International NV, IBM Corporation, Imagination Technologies, Creative Labs, NVIDIA
Portable
Works on heterogeneous architecture
Provides great computational power
Okay so, OpenCL is a low level library for writing programs that execute across heterogeneous systems consisting of CPU’s, GPU’s, FPGA’s, CELL processors.
This is basically a modern computer, which may have 2 or more of these devices.
The vendor is responsible to support OpenCL, these are the current supporting vendors:
Vivante Corporation – embedded GPUs, STMicroelectronics International NV, IBM Corporation, Imagination Technologies – mobile grapics systems
ARM, AMD, Intel, IBM, already the big names in hardware manufacturing.
You could call OpenCL portable, because you can write the same executable code which can be run for any of these devices, for any operating system
Works on heterogeneous architecture – windows, linux – the same code!
The GPU in particular has seen great computational power speedup which I will show you in the next slide versus a CPU

4. How much computational power?
Speed of GPU’s are growing rapidly vs CPU’s
With all of this potential power on a personal computer, why not use it?
Many of us here use the grid to run large computations, this is also a possible option.
So the question is, why is this so much faster?
A GPU has hundreds of computational cores which are capable of running multiple threads, versus the CPU’s two or four cores, with one or two threads per core.
A GPU is designed to run a lot of simple calculations concurrently.

5. Key Concepts OpenCL - Runtime system Kernel Accelerated Device
Runtime system OpenCL is a runtime system.
When the program runs, it compiles OpenCL Kernel code.
This Kernel, is eventually the piece of code that will be run on the accelerated device. I will give an example what this looks like compared to standard code.
Because this kernel will be compiled during runtime, any device can be used on any system.
An OpenCL supported accelerated device is a dedicated processing device that is capable of running OpenCL kernels.

6. OpenCL Kernel
// Sequential c/c++ code for( int x = 0; x < 1024; x++ ) { for( int y = 0; y < 1024; y++ ) matrix[x][y] = matrix[x][y] + 1; // Code is run times for this thread } // Parallel kernel code kernel void MatrixIncrement( global int** matrix) int x = get_global_id(0); int y = get_global_id(1); matrix[x][y] = matrix[x][y] + 1; // Code is run once for this thread
Say we have a matrix of width 1024 and height 1024
And we want to simply increment the value.
In a normal sequential style it will be programed as folows,
With two loops we will iterate over indexes x and y, and increment that value. Simple as that, however, that line of code will be run more than a million times, one after the other.
In the parallel kernel code, a function will be executed by the device.
Here, every thread will perform this code.
It will retrieve its unique index position, and run the increment once. Done.

7. Problems Low level C/C++ Library A lot of overhead code
Things can and will go wrong
Now this may seem all simple and easy, but lets not forget,
We are dealing with a low level library, where the programmer has to handle a lot of things.
A lot of programmers do not want to get tied up in the hassle of managing the devices we are working on, managing the code and time it compiles or runs, et cetera.
This is overhead code that reduces productivity, and interest in using this library.
Because the programmer has to deal with so much, things will go wrong, they always do.
Lets look at a model that represents ease of application development of opencl applications
Wrappers pycl, cloo, etc.

8. Ease of OpenCL application development
Increase ease of application development
Drivers and Hardware, CPU’s, GPU’s, Cell Processors, FPGA’s
OpenCL C Library
Tools:
Debuggers,
Profilers,
High Level Frameworks
Middleware/Library:
Video, Imaging,
Math/Sciences,
Physics
Wrappers C++, C#, Java, Python, Javascript
At the lowest level, the drivers communicate with the hardware. The programmer does not deal with this.
On top of this, the opencl library communicates with the drivers to compile and run kernels.
Here the programmer needs to handle compiling the code, requesting available devices, creating memory specifically for a device, etc.
Tools have been created for the library that can debug it, profile it. Very handy tools
There are some lower middleware and libraries to do some video, imaging math or physics.
On top of that there are high level frameworks that seamlessly handle these layers to increase ease of application development.
This is the scope of my project, and I aim to make it easier for programmers to write OpenCL code.
We are going to take a look at the high level frameworks section, where a lot of research has been done in optimizing, and making it easier to write opencl code.

9. High Level Frameworks Research has been done in:
Scheduling multiple kernels on device
Overlapping memory transfers with kernel execution
Load balancing
Distribution over GPU’s on the grid
Task scheduling
Various research has been done in optimizing the use of an accelerated device, this is a bit more technical
Running multiple kernels on the same device at the same time yielded in some performance enhancement.
Transferring memory to a device and running a kernel on it at the same time.
Load balancing between different accelerated devices.
Running opencl code on the grid
Load balancing pci express bus memory transfer..

10. Dataflow Processing Framework
In a nutshell:
Based on ideas of different research
Increase the ease of development
Uses the dataflow concept
Simplicity
Asynchronous overlapped data transfers and kernel executions
The research puts forward some interesting ideas, however I start to see the lack of ease of development, conceptual development of an algorithm, so I designed a framework that brings together a lot of ideas.
In a nutshell, I wanted to make the framework to increase the ease of development,
Use the dataflow concept, useful for conceptual programming
For performance I do asynchronous overlapped data transfer and kernel executions
Keep it simple
For a programmer

11. Conceptual Example 1 2 3 4 5 6 Input A Input B
Legend: Async Process Async memory xfer CPU Process GPU Process Data Dependency Data Output 1 2 3 4 5
Lets look at a conceptual example of a dataflow processing scenario
When I look at an algorithm I see some data being processed in some way, and it can be broken down into a flow.
A conceptual example shows a lot more
Here we see six asynchronous processes, each will require input data, and produce output data.
Some processes must wait for others, and the data must be prepared for them
Here we have two inputs, which will be needed by the CPU and GPU.
This data will get sent to the appropriate devices in parallel.
Once the data is ready for those processes they can start.
Etc.
This all happens independently and overlapped. 6
Output A
Output B

12. Programming with the Framework
Programmer defines a number of processes and data
The process uses a OpenCL kernel or a standard C/C++ function
User defines the arguments of the kernel with the defined data
These processes compute on user selected device: CPU/ GPU/FPGA… etc
Signal the framework to run

13. Programming Example Framework 1 Sort 2 Sort 3 Filter 4 Search Array A
ProcessingFramework pf; ProcessingComponent one, two, three, four; DeviceMemory ArrayA, ArrayB, Output; ArrayA = pf.CreateInputMemory( mem_size ); ArrayB = pf.CreateInputMemory( mem_size ); Output = pf.CreateOutputMemory( mem_size ); one = pf.CreateAPC( pf.GPUDevice(), "Sort" ); two = pf.CreateAPC( pf.CPUDevice(), "Sort" ); three = pf.CreateAPC( pf.GPUDevice(), "Filter" ); four = pf.CreateAPC( pf.CPUDevice(), "Search" ); one.SetArg( 0, ArrayA ); one.SetWorkSize( arr_size ); two.SetArg( 0, ArrayB ); two.SetWorkSize( arr_size ); three.SetDependency( one, 0, ArrayA ); three.SetDependency( two, 1, ArrayB ); three.SeWorkSize( arr_size ); four.SetDependency( three, 0, ArrayA ); four.SetArg( 1, Output ); four.SetWorkSize( arr_size ); pf.Run();
Array A
Array B
1 Sort
2 Sort
3 Filter
4 Search
Output