REU WEEK III Malcolm Collins-Sibley Mentor: Shervin Ardeshir

PROJECT Cross-View Image Registration and Semantic Segmentation The goal is to use information from map and satellite images, and project them on the screen which the user is observing, in a way that the user can see semantic segments overlaid on the scene.

PROJECT Output Mockup

COMPLETED WORK Readings: “Geometric Image Parsing in Man-Made Environments” Olga Barinova et al “Recovering Surface Layout from an Image” Derek Hoiem et al “Recovering Occlusion Boundaries from a Single Image” Derek Hoeim et al “Entropy Rate Superpixel Segmentation” MY Liu et al

COMPLETED WORK Geometric Image Parsing Code

COMPLETED WORK Geometric Image Parsing Code

COMPLETED WORK Super-pixel Segmentation With 8 super-pixels

COMPLETED WORK Super-pixel Segmentation With 20 super-pixels

COMPLETED WORK Building Projection

COMPLETED WORK Building Projection

CURRENT WORK Within the Building Projection code: Building occlusion and self-occlusion Works when a building occludes another, but not when a building is occluding itself

CURRENT WORK Occlusion Handling BeforeAfter

CURRENT WORK Occlusion Handling Before After

CURRENT WORK Occlusion Handling BeforeAfter

CURRENT WORK Occlusion Handling BeforeAfter

THE NEXT STEP Understanding the occlusion handling code Making sure it is handling self-occlusions accurately Understanding the format of the output data in the line segments/horizon code Running the line segmentation code for all of the images in our dataset and saving all of the output variables in a structure Extracting the super pixels from images in the dataset and saving it in a structure Computing their pairwise similarities of the super pixels in terms of color and texture