Peter Tummeltshammer, Martin Delvai HW-SW Codesign Lab Course Setup Peter Tummeltshammer, Martin Delvai http://ti.tuwien.ac.at/ecs/teaching/courses/hwswcode_lu

Overview Face recognition Implementation details Further Steps How does it work Lab course approach Implementation details Further Steps ECS Group Tummeltshammer & Delvai

Facial Recognition System Computer-driven application Identifies a person from a digital image Identification vs. Verification Identification: Comparing to a single image Verification: Comparing to a database Not to confuse with face detection Locate a face inside an image ECS Group Tummeltshammer & Delvai

Known FR-Algorithms 2D Face Recognition 3D-Face recognition Eigenfaces Correlation filters Hidden Markov Model Dynamic Link Matching 3D-Face recognition ECS Group Tummeltshammer & Delvai

Image Correlation 2D FFT / IFFT Point-wise multiplication Peak to Sidelobe Ratio (PSR) calculation ECS Group Tummeltshammer & Delvai

Input Images 32x32 pixels 8 bit greyscale No/only little background Consistent lightning High contrast ECS Group Tummeltshammer & Delvai

… … 2D FFT decomposition Can be decomposed into 1D FFTs 1. 1D 32-point FFT of all 32 lines 2. 1D 32-point FFT of all 32 columns Overall 64 32-point FFTs … … ECS Group Tummeltshammer & Delvai

2D FFT result Result: Complex 32x32 Matrix Problem for fixed-point precision: big numerical range real imag ECS Group Tummeltshammer & Delvai

2D FFT Decomposes image data into frequency domain Representation of facial feature transitions Constant component at (0,0) ECS Group Tummeltshammer & Delvai

Pointwise Multiplication Complex point-wise multiplication (a+bi) * (c+di)=ac+adi+bci-bd Four Multiplications for each pixel Overall 4096 Multiplications ECS Group Tummeltshammer & Delvai

2D inverse FFT (IFFT) Similar to 2D FFT (conjugated complex) 64 32-point 1D IFFTs Only real part necessary for PSR calculation ECS Group Tummeltshammer & Delvai

Peak to Sidelobe Ratio PSR is a measure for correlation 1. Find Peak 2. draw 5x5 and 21x21 rectangles (sidelobe region) 3. calculate µ and σ inside sidelobe region 21 5 32 32 ECS Group Tummeltshammer & Delvai

Peak to Signal Ratio High PSR = high correlation PSR > Θ -> Person identified Θ is application- and filter specific PSR=38 PSR=3 ECS Group Tummeltshammer & Delvai

Correlation in Face Recognition Offline Generation of reference filter One correlation filter for each person Out of N training images Filters are stored in frequency domain Online Offline ECS Group Tummeltshammer & Delvai

Verification Example PSR 1 ≤ Θ PSR 2 ≤ Θ … PSR N > Θ VerifyImage(image, filterdatabase) { fftimage = FFT(image); for (i=0; i<size(filterdatabase); i++) { multimage = fftimage * filterdatabase[i]; ifftimage = ifft(multimage); PSR = computePSR(ifftimage); if (PSR > Θ) { return i; } return -1; PSR 1 ≤ Θ PSR 2 ≤ Θ … PSR N > Θ ECS Group Tummeltshammer & Delvai

MACE Filter Design Minimum Average Correlation Energy Filter [Savvides et al.] High illumination invariance Generates sharp peaks in PSR analysis Filter generation performed offline For details: [Savvides et al.] or MATLAB ECS Group Tummeltshammer & Delvai

  Design Path MATLAB Model Algorithmic approach Floating point precision Provided by us C- Implementation Fixed point precision Provided by us HWSW-CD Partitioning Timing optimization Todo by you ECS Group Tummeltshammer & Delvai

MATLAB Model Available on HW-SW CD Homepage Optional (only if interested) Workspace (facerec.mat) Images (10 example images) Filter (single correlation Filter) Functions computePSR generateFilter ECS Group Tummeltshammer & Delvai

MATLAB Model Example command: computepsr(real(ifft2(fft2(double(Images(:,:,:,i))) .* filterN))); Computes PSR of filterN and Image i inside Vector “Images“ Ifft2 and fft2 are MATLAB functions that need double precision input Computepsr returns PSR of given matrix ECS Group Tummeltshammer & Delvai

C Implementation Available on HW-SW CD Homepage Mandatory download Calculates PSR for given image and filter Uses 16 bit fixed point representation ECS Group Tummeltshammer & Delvai

C Implementation Uses fixed point representation Challenges: Most arithmetic operations can cause an overflow Requires scaling Fixed scaling -> loss of precision Pro: dynamic scaling won‘t affect the PSR calculation due to division by standard deviation ECS Group Tummeltshammer & Delvai

C Filestructure Compile with make (gcc) /facerec frconsole.c fix_fft.c Makefile filter_2_4_9.txt image1.txt image2.txt image10.txt … Compile with make (gcc) Filter generated from images 2, 4 and 9 (same as MATLAB) Syntax: ./frconsole image1.txt filter_2_4_9.txt ./frconsole image2.txt filter_2_4_9.txt … ECS Group Tummeltshammer & Delvai

C Implementation FFT taken from web (http://www.jjj.de/) FFT scales automatically PW multiplication and PSR calculation scale dynamically Images and filter are scaled to 16 bit integer [-32768, 32767] ECS Group Tummeltshammer & Delvai

HW-SW CD Partitioning Implemented on ALTERA FPGA, using Softcore SPEAR 16 bit Instruction & Data Memory No FPU, no Multiplication Integration of custom HW-modules by memory mapping Concept presentation on 14.5.07 ECS Group Tummeltshammer & Delvai

HW-SW CD Partitioning Timing optimization by Analyzing code for repetative and time- consuming function calls Implementing HW-modules for these functions Implementing whole blocks (i.e. like 2D- FFT) in HW ECS Group Tummeltshammer & Delvai

Further Steps 1.: Download and understand C-Code 2.(optional): Download MATLAB design 3.: Compile C-code for SPEAR (SPEAR concept presentation: 14.5.07) 4.: Analyze time consumption 5.: Implement HW modules and partition design ECS Group Tummeltshammer & Delvai

Questions? ECS Group Tummeltshammer & Delvai

Organisatorisches Gruppenbildung Email in TI System Fragen zu C-Code: Mi 16.5.07 10:00 – 14:00 ECS Group Tummeltshammer & Delvai