1. Device Routines and device variables
These notes introduce:
Declaring routines that are be executed on device and on the host
Declaring local variable on device
ITCS 4/5145 Parallel Programming, B. Wilkinson, July 11, CUDADeviceRoutines.ppt

2. CUDA extensions to declare kernel routines
Host = CPU Device = GPU
__global__ indicates routine can only be called from host and only executed on device
__device__ indicates routine can only be called from device and only executed on device
__host__ indicates routine can only be called from host and only executed on host
(generally only used in combination with __device__ , see later)
Two underscores each
Note: Cannot call a routine from the kernel to be executed on host

3. So far we have seen __global__:…
__global__ void add(int *a,int *b, int *c) {
int tid = blockIdx.x * blockDim.x + threadIdx.x;
if(tid < N) c[tid] = a[tid]+b[tid]; }
int main(int argc, char *argv[]) {
int T = 10, B = 1; // threads per block and blocks per grid
int a[N],b[N],c[N];
int *dev_a, *dev_b, *dev_c;
cudaMalloc((void**)&dev_a,N * sizeof(int));
cudaMalloc((void**)&dev_b,N * sizeof(int));
cudaMalloc((void**)&dev_c,N * sizeof(int));
cudaMemcpy(dev_a, a , N*sizeof(int),cudaMemcpyHostToDevice);
cudaMemcpy(dev_b, b , N*sizeof(int),cudaMemcpyHostToDevice);
cudaMemcpy(dev_c, c , N*sizeof(int),cudaMemcpyHostToDevice);
add<<<B,T>>>(dev_a,dev_b,dev_c);
cudaMemcpy(c,dev_c,N*sizeof(int),cudaMemcpyDeviceToHost);
cudaFree(dev_a);
cudaFree(dev_b);
cudaFree(dev_c);
cudaEventDestroy(start);
cudaEventDestroy(stop);
return 0;
__global__ must have void return type. Why?
Executed on device
Called from host
Note __global__ asynchronous.
Returns before complete

4. Routines to be executed on device
Generally cannot call C library routines from device!
However CUDA has math routines for device that are equivalent to standard C math routines with the same names, so in practice can call math routines such as sin(x) – need to check CUDA docs* before use
Also CUDA has GPU-only routines implemented, faster less accurate (have __ names)*
* See NVIDIA CUDA C Programming Guide for more details

5. __device__ routines __global__ void gpu_sort (int *a, int *b, int N) {…
swap (&list[m],&list[j]); }
__device__ void swap (int *x, int *y) {
int temp;
temp = *x;
*x = *y;
*y = temp;
int main (int argc, char *argv[]) {
gpu_sort<<< B, T >>>(dev_a, dev_b, N);
return 0;
Recursion is possible with __device __ routines so far as I can tell

6. Routines executable on both host and device
__device__ and __host__ qualifiers can be used together
Then routine callable and executable on both host and device. Routine will be compiled for both.
Feature might be used to create code that optionally uses a GPU or for test purposes.
Generally will need statements that differentiate between host and device
Note: __global__ and __host__ qualifiers cannot be used together.

7. __CUDA_ARCH__ macro Indicates compute capability of GPU being used.
Can be used to create different paths thro device code for different capabilities.
__CUDA_ARCH__ = 100 for 1.0 compute capability
__CUDA_ARCH__ = 110 for 1.1 compute capability …

8. Example __host__ __device__ func() { #ifdef __CUDA_ARCH__
… // Device code
#else
… // Host code
#endif }
Could also select specific compute capabilities

9. Declaring local variables for host and for device

10. Local variables on host
#include <stdio.h>
#include <stdlib.h>
int cpuA[10];
...
void clearArray() {
for (int i = 0; i < 10; i++)
cpuA[i] = 0; }
void setArray(int n) {
cpuA[i] = n;
int main(int argc, char *argv[]) { …
clearArray();
setArray(N);
return 0;
Local variables on host
In C, scope of a variable is block it is declared in, which does not extend to routines called from block.
If scope is to include main and all within it, including called routines, place declaration outside main:

11. Declaring local kernel variables
#include <stdio.h>
#include <stdlib.h>
__device__ int gpu_A[10];
...
__global__ void clearArray() {
for (int i = 0; i < 10; i++)
gpuA[i] = 0; }
int main(int argc, char *argv[]) { …
clearArray<<<…>>>();
setArray<<<…>>>(N);
return 0;
Declaring variable outside main but use __device__ (now used as a variable type qualifier rather than function type qualifier)
Without further qualification, variable is in global (GPU) memory.
Accessible by all threads

12. Accessing kernel variables from host
Accessible by host using routines:
cudaMemcpyToSymbol() to copy to device
cudaMemcpyFromSymbol() to copy from device
where name of kernel variable given as a String argument:
int main(int argc, char *argv[]) {
int cpuA[10]: …
cudaMemcpyFromSymbol(&cpuA, "gpuA", sizeof(cpuA), 0,
cudaMemcpyDeviceToHost);
return 0; }
Contents of gpuA[] copied to cpuA[]
Name of variable declared in kernel

13. Example of both local host and device variables
#include <stdio.h>
#include <cuda.h>
#include <stdlib.h>
int cpu_hist[10]; // accessible on cpu only
__device__ int gpu_hist[10]; // accessible on gpu only
void cpu_histogram(int *a, int N) {
… // result in cpu_hist[] }
__global__ void gpu_histogram(int *a, int N) {
… // result in gpu_hist[]
int main(int argc, char *argv[]) { …
gpu_histogram<<<B,T>>>(dev_a,N); // computed on gpu
cpu_histogram(a,N); // computed on cpu
return 0;
Example of both local host and device variables

14. Questions