Application of light fields in computer vision AMARI LEWIS – REU STUDENT AIDEAN SHARGHI- PH.D STUENT

Main objective  increase object recognition through using the EPI of light field images  Using the light field camera

Using the Lytro light field camera  conventional methods- involve using 2D information  Light field images- captures all 3D information in a single shot.  Using the Lytro light field camera to collect dataset  camera captures light field direction, intensity and color

Datasets- 1. Collected own dataset using the Lytro light field camera ◦Bikes ◦Buildings ◦Trees ◦Vehicles - Studying the 7 different image perspectives

2. Dataset from Switzerland using the iphone video ◦Buildings – 50 categories -Ranging from 4-30 videos -Extracted 300 frames from each video

Epipolar planar images- EPI  It is a 2D representation or slice of an image  Taking the same line from each image and putting it on top of each other  Using the multiple shots taken from the camera and the extracted frames

Light field 7 lines from each of the images concatenated- total of 1080

Concatenated the 300 lines – total 720

Implementing DCT Steps: Separate the RGB into 3 channels Calculate the row-wise mean- calculates the mean of each row to create a vector Calculate the DCT for each channels Concatenate some coefficients, using as a feature vector (smaller)

For classification Apply Principal component analysis (PCA) gmm- Gaussian mixture model Linear SVM

Best Results Using this method on EPIs ◦Lytro Light field camera dataset 77% accuracy Switzerland dataset 96% accuracy

Thank you