1. An Introduction to GPU Computing
Ryan Szypowski
Department of Mathematics
and Statistics

2. Outline
Who am I?
Why GPU?
General Model
OpenCL
Example

3. What are GPUs good for?

4. What are GPUs good for?

5. What are GPUs good for?

6. What are GPUs good for?

7. What are GPUs REALLY good for?
Lots and lots of independent calculations
Specifically, the same “kernel” of computation on independent “streams” of data
Development is driven by the deep-pocketed gaming industry
Computing and memory bandwidth gap between CPU and GPU is widening

8. Shameless theft
(stolen from a talk by Wen-mei Hwu and John Stone and credited to John Owens)

9. General Model Based on “stream” processing
A stream is a set of records requiring the same processing
Data parallelism or loop-level parallelism
The processing is called a “kernel”
In graphics applications, the streams are vertices and fragments, and the kernels are vertex and fragment shaders

10. General Model
Data inside a kernel is either input or output, never both
Ideal GPU computing has high amounts of data which requires a significant amount of computing, but that the computing is as independent as possible

11. OpenCL
(stolen from the Khronos Group talk by Ofer Rosenberg)

12. OpenCL
OpenCL has a single host program (standard C code or preferred language)
The computation to be parallelized is called a work- item
The code for a work item is stored in a kernel
For platform independence, the kernels are compiled at run-time

13. OpenCL Work-items are grouped into workgroups
Work-items within a workgroup execute simultaneously
Workgroups are scheduled asynchronously
Workgroups can be organized in different topologies
Memory model is complicated and must be dealt with explicitly

14. OpenCL

15. OpenCL: Basic Structure
Create a “Context”
Basically something that contains all the other structures
Get the “Device(s)” that you will work on
Create a “Command Queue” in your context

16. OpenCL: Basic Structure
Allocate memory “Buffers” in your context
Compile your “Kernel”
Copy data from host memory into device memory
“Execute” the kernel!
Copy results back

17. Example
Code modified (very slightly) from Erik Smistad's blog post about OpenCL
Vector addition
(CPU)
for i = 1:n
c[i] = a[i] + b[i]
(GPU)
i = get_global_id(0)
c[i] = a[i] + b[i]

18. Example: vector_add_kernel.cl
__kernel void vector_add(__global int *A,
__global int *B, __global int *C) {
// Get the index of the current element
int i = get_global_id(0);
// Do the operation
C[i] = A[i] + B[i]; }

19. Example: main.c (portions)
// Get platform and device information
cl_platform_id platform_id = NULL;
cl_device_id device_id = NULL;
cl_uint ret_num_devices;
cl_uint ret_num_platforms;
cl_int ret = clGetPlatformIDs(1, &platform_id,
&ret_num_platforms);
ret = clGetDeviceIDs( platform_id, CL_DEVICE_TYPE_ALL,
1, &device_id, &ret_num_devices);

20. Example: main.c (portions)
// Create an OpenCL context
cl_context context = clCreateContext( NULL,
1, &device_id, NULL, NULL, &ret);
// Create a command queue
cl_command_queue command_queue =
clCreateCommandQueue(context,
device_id, 0, &ret);

21. Example: main.c (portions)
// Create memory buffers on the device for each vector
cl_mem a_mem_obj = clCreateBuffer(context,
CL_MEM_READ_ONLY,
LIST_SIZE * sizeof(float), NULL, &ret);
cl_mem b_mem_obj = clCreateBuffer(context,
cl_mem c_mem_obj = clCreateBuffer(context,
CL_MEM_WRITE_ONLY,

22. Example: main.c (portions)
// Copy the lists A and B to respective memory buffers
ret = clEnqueueWriteBuffer(command_queue,
a_mem_obj, CL_TRUE, 0,
LIST_SIZE * sizeof(float), A, 0, NULL, NULL);
b_mem_obj, CL_TRUE, 0,
LIST_SIZE * sizeof(float), B, 0, NULL, NULL);

23. Example: main.c (portions)
// Build the program and create the OpenCL kernel
ret = clBuildProgram(program, 1, &device_id, NULL, NULL, NULL);
cl_kernel kernel = clCreateKernel(program, "vector_add", &ret);
// Set the arguments of the kernel
ret = clSetKernelArg(kernel, 0, sizeof(cl_mem), (void *)&a_mem_obj);
ret = clSetKernelArg(kernel, 1, sizeof(cl_mem), (void *)&b_mem_obj);
ret = clSetKernelArg(kernel, 2, sizeof(cl_mem), (void *)&c_mem_obj);

24. Example: main.c (portions)
// Execute the OpenCL kernel on the list
size_t global_item_size = LIST_SIZE;
size_t local_item_size = 64; // groups of 64
ret = clEnqueueNDRangeKernel(command_queue,
kernel, 1, NULL, &global_item_size,
&local_item_size, 0, NULL, NULL);

25. Example: main.c (portions)
// Read the memory buffer C on the device to
// the local variable C
int *C = (int*)malloc(sizeof(float)*LIST_SIZE);
ret = clEnqueueReadBuffer(command_queue,
c_mem_obj, CL_TRUE, 0,
LIST_SIZE * sizeof(float), C, 0, NULL,
NULL);

26. Results Run on home desktop Running Fedora 17
Intel i (quad core + hyperthreading)
NVIDIA GeForce GTX 570 (480 CUDA cores)

27. Results

28. References http://www.khronos.org/opencl/
ncl_lec1.pdf
opencl-and-gpu-computing/

29. Thanks!