Finding Body Parts with Vector Processing Cynthia Bruyns Bryan Feldman CS 252

Introduction Take existing algorithm for tracking human motion, speed up by computing on the GPU. Demonstrate that many vision algorithms are prime candidates for using vector processing

Results after false candidates have been removed Demo

Vision Algorithms Often computationally expensive- searching over many pixels for objects at many orientations and scales E.g. [((1024x768)pix)x3colors]x[12orientations]x[5 scales] Very often the case that highly parallizable

Limb Finding Goal – find candidate limbs Limbs look like long dark rectangles on light backgrounds or long light things on dark backgrounds

1. Convolution with filter convolve using FFT Response indicates how much pixels go from low to high intensity Convolve over all three color channels so as to not miss red – blue of same intensity Algorithm specifics * x

2. For every pixel location get resp conv from “left” and “right”, put into new matrix resp limb Algorithm specifics -resp conv x x resp conv x x resp limb

Algorithm specifics 3. Find local maximums – for every pixel replace with max. of local neighbors. If resp limb =locMax it’s a max resp limb locMax

GPU It’s a good choice because each operation is per pixel – SIMD-like Data stored in texture buffers equivalent to local cache Clean instruction set and developing interface language to exploit vector operations Justify your gaming habits

GPU dataflow model Hardware supports several data types for bandwidth optimization, i.e. 32 bit floating point, half etc. Data passed to main memory stages via binding Application Fragment Processor Assembly & Rasterization Framebuffer Operations Framebuffer Textures Vertex Processor

Fragment processor has high resource limits 1024 instructions 512 constants or uniform parameters Each constant counts as one instruction 16 texture units Reuse as many times as desired No branching But, can do a lot with condition codes No indexed reads from registers Use texture reads instead No memory writes

The algorithm Draw invokes the fragment programs The texture becomes a data structure – use two for framebuffers to avoid RAW hazzards FFT Fragment program Image Mask Convolution Program Cylinder Program Find Max Program For each orientation to search

Results (CPU-2.53 GHz P4 GPU Nvidia FX5900) Mask size fixed (22x13) vary image size *Additional GPU optimizations possible

Results – log scale (CPU-2.53 GHz P4 GPU Nvidia FX5900) Mask size fixed (22x13) vary image size 42.7 sec sec *Additional GPU optimizations possible

Results Image size fixed (512x512) vary mask size Varying mask sizes allow for varying limb sizes on same image

Results

Comments GPU and image processing are a good match Time to move memory from CPU to GPU is cumbersome – but can be overcome Non-uniformity of installations, products, exact specifications are hearsay

Acknowledgements Kenneth Moreland Deva Ramanan Okan Arikan