Window Filling and Junk Data Removal Hadi Fadaifard.

Slides:



Advertisements
Similar presentations
QR Code Recognition Based On Image Processing
Advertisements

CS 691 Computational Photography Instructor: Gianfranco Doretto 3D to 2D Projections.
Morphological Image Processing Md. Rokanujjaman Assistant Professor Dept of Computer Science and Engineering Rajshahi University.
Image and View Morphing [Beier and Neely ’92, Chen and Williams ’93, Seitz and Dyer ’96]
Camera Calibration. Issues: what are intrinsic parameters of the camera? what is the camera matrix? (intrinsic+extrinsic) General strategy: view calibration.
A Versatile Depalletizer of Boxes Based on Range Imagery Dimitrios Katsoulas*, Lothar Bergen*, Lambis Tassakos** *University of Freiburg **Inos Automation-software.
Camera calibration and epipolar geometry
Announcements Final Exam May 16 th, 8 am (not my idea). Practice quiz handout 5/8. Review session: think about good times. PS5: For challenge problems,
Announcements. Projection Today’s Readings Nalwa 2.1.
X86 and 3D graphics. Quick Intro to 3D Graphics Glossary: –Vertex – point in 3D space –Triangle – 3 connected vertices –Object – list of triangles that.
CS223B Assignment 1 Recap. Lots of Solutions! 37 Groups Many different approaches Let’s take a peek at all 37 results on one image from the test set.
Announcements Mailing list (you should have received messages) Project 1 additional test sequences online Talk today on “Lightfield photography” by Ren.
CS485/685 Computer Vision Prof. George Bebis
Gopi -ICS186AW03 - Slide1 Graphics Pipeline: First Pass.
Announcements Mailing list Project 1 test the turnin procedure *this week* (make sure it works) vote on best artifacts in next week’s class Project 2 groups.
CSSE463: Image Recognition Day 30 Due Friday – Project plan Due Friday – Project plan Evidence that you’ve tried something and what specifically you hope.
The Pinhole Camera Model
A Novel 2D To 3D Image Technique Based On Object- Oriented Conversion.
Part I: Basics of Computer Graphics Rendering Polygonal Objects (Read Chapter 1 of Advanced Animation and Rendering Techniques) Chapter
October 8, 2013Computer Vision Lecture 11: The Hough Transform 1 Fitting Curve Models to Edges Most contours can be well described by combining several.
FEATURE EXTRACTION FOR JAVA CHARACTER RECOGNITION Rudy Adipranata, Liliana, Meiliana Indrawijaya, Gregorius Satia Budhi Informatics Department, Petra Christian.
Development of Image Processing Based Feedback Systems for Interactive Gaming Using Non-Traditional Controllers Adam Hedji Mantas Pulinas Philip San III.
Computer Graphics Inf4/MSc Computer Graphics Lecture 9 Antialiasing, Texture Mapping.
Technology and Historical Overview. Introduction to 3d Computer Graphics  3D computer graphics is the science, study, and method of projecting a mathematical.
October 14, 2014Computer Vision Lecture 11: Image Segmentation I 1Contours How should we represent contours? A good contour representation should meet.
CSSE463: Image Recognition Day 30 This week This week Today: motion vectors and tracking Today: motion vectors and tracking Friday: Project workday. First.
02/26/02 (c) 2002 University of Wisconsin, CS 559 Last Time Canonical view pipeline Orthographic projection –There was an error in the matrix for taking.
Advanced Computer Graphics Three Dimensional Viewing
Week 5 - Wednesday.  What did we talk about last time?  Project 2  Normal transforms  Euler angles  Quaternions.
Sebastian Thrun CS223B Computer Vision, Winter Stanford CS223B Computer Vision, Winter 2005 Lecture 2 Lenses and Camera Calibration Sebastian Thrun,
CS 638, Fall 2001 Today Project Stage 0.5 Environment mapping Light Mapping.
CS-378: Game Technology Lecture #2.1: Projection Prof. Okan Arikan University of Texas, Austin Thanks to James O’Brien, Steve Chenney, Zoran Popovic, Jessica.
Computer Graphics Bing-Yu Chen National Taiwan University.
Rendering Overview CSE 3541 Matt Boggus. Rendering Algorithmically generating a 2D image from 3D models Raster graphics.
Course 13 Curves and Surfaces. Course 13 Curves and Surface Surface Representation Representation Interpolation Approximation Surface Segmentation.
Gopi -ICS280F02 - Slide 1 Graphics Pipeline: First Pass.
3D Photorealistic Modeling Process. Different Sensors Scanners Local coordinate system Cameras Local camera coordinate system GPS Global coordinate system.
CS-498 Computer Vision Week 7, Day 2 Camera Parameters Intrinsic Calibration  Linear  Radial Distortion (Extrinsic Calibration?) 1.
CS 450: COMPUTER GRAPHICS PROJECTIONS SPRING 2015 DR. MICHAEL J. REALE.
Computer Graphics Zhen Jiang West Chester University.
OpenGL The Viewing Pipeline: Definition: a series of operations that are applied to the OpenGL matrices, in order to create a 2D representation from 3D.
Basic Perspective Projection Watt Section 5.2, some typos Define a focal distance, d, and shift the origin to be at that distance (note d is negative)
Computer Graphics Lecture 08 Fasih ur Rehman. Last Class Ray Tracing.
Segmentation of Vehicles in Traffic Video Tun-Yu Chiang Wilson Lau.
Lecture 6 Rasterisation, Antialiasing, Texture Mapping,
Foundations of Computer Graphics (Spring 2012) CS 184, Lecture 5: Viewing
Hough Transform Procedure to find a shape in an image Shape can be described in parametric form Shapes in image correspond to a family of parametric solutions.
Rendering Pipeline Fall, D Polygon Rendering Many applications use rendering of 3D polygons with direct illumination.
Viewing and Projection
CSSE463: Image Recognition Day 29 This week This week Today: Surveillance and finding motion vectors Today: Surveillance and finding motion vectors Tomorrow:
Image Segmentation Image segmentation (segmentace obrazu)
Coordinate Systems Lecture 1 Fri, Sep 2, The Coordinate Systems The points we create are transformed through a series of coordinate systems before.
Structure from Motion Paul Heckbert, Nov , Image-Based Modeling and Rendering.
Detection of closed sharp edges in point clouds Speaker: Liuyu Time:
Robotics Chapter 6 – Machine Vision Dr. Amit Goradia.
Viewing and Projection. The topics Interior parameters Projection type Field of view Clipping Frustum… Exterior parameters Camera position Camera orientation.
Computer Vision Computer Vision based Hole Filling Chad Hantak COMP December 9, 2003.
Quality of Images.
Dynamic View Morphing performs view interpolation of dynamic scenes.
The Camera : Computational Photography
3D Clipping.
Mean Shift Segmentation
3D Rendering Pipeline Hidden Surface Removal 3D Primitives
OpenGL 2D Viewing Pipeline
CSL 859: Advanced Computer Graphics
Binary Image processing بهمن 92
Filtering Things to take away from this lecture An image as a function
ECE 692 – Advanced Topics in Computer Vision
Finding Line and Curve Segments from Edge Images
Filtering An image as a function Digital vs. continuous images
Presentation transcript:

Window Filling and Junk Data Removal Hadi Fadaifard

The goal of this project was to fill windows/holes that are present in the 3D scans of large buildings. It lead to realization of the need to remove junk data: It lead to realization of the need to remove junk data: Points behind the windows Points from other objects not related to the building: Trees Trees Traffic lights Traffic lights Flags Flags

Some Failed Techniques Linear Interpolation: Fill the holes by interpolating in the place of missing points Problems: Problems: Junk data inside the windows affect and distort the generated points. Edges and occlusion make the result of interpolation unpleasant.

2D to 3D projection Fill Holes in the 2D image of the building and project the results back in 3D: Do segmentation of the points in the scan and find the planes each point belongs to. Do segmentation of the points in the scan and find the planes each point belongs to. For each hole region, find the plane its neighbors belong to. For each hole region, find the plane its neighbors belong to.

2D to 3D projection Find the intrinsic parameters of the scanner. (I assumed it could be modeled with a pinhole camera.) Find the intrinsic parameters of the scanner. (I assumed it could be modeled with a pinhole camera.) Do reverse perspective projection from 2D into 3D. Do reverse perspective projection from 2D into 3D.

2D to 3D projection The results were better than the previous techniques Correct projection was not obtained due to: Inaccuracies in finding the destination plane in 3D because of junk data. Inaccuracies in finding the destination plane in 3D because of junk data. Did not take into account the large radial distortion of the scanner. Did not take into account the large radial distortion of the scanner.

Removing Junk Data Junk data always affect the results of window filling algorithms. Remove as much junk data as possible before hole filling.

Junk Removal Method I: Find the top planes that define the building. Find the top planes that define the building. Determine the plane each point belongs to. Determine the plane each point belongs to. Consider the points that are not in any plane as junk. Remove them Consider the points that are not in any plane as junk. Remove them

Junk Removal Problems with this Method: It would consider some junk data as legitimate (e.g. ceilings) It would consider some junk data as legitimate (e.g. ceilings) It would remove lots of useful data (e.g. around the window edges) It would remove lots of useful data (e.g. around the window edges)

Junk Removal Find top 4-5 planes and remove all points that don’t belong any of the found planes AND have are behind the planes. Problem: it would still regard some junk as useful data. Problem: it would still regard some junk as useful data.

A Solution that works Algorithm: 1. Find the planes that define the building 2. Do orthographic projection of all points near the plane (<t). 3. Generate an image of the projected points. 4. Use connected component algorithm to find and fill the windows in 2D. 5. Transform the filled windows back to the original 3D coordinate system.

Finding the planes Hough transform (or Accumulator?): Find the normal at each point: Find the normal at each point: Transform the normal to spherical coordinate system and store the θ, φ in a 2D accumulator. Generate an image of the accumulator. Transform the normal to spherical coordinate system and store the θ, φ in a 2D accumulator. Generate an image of the accumulator.

Finding Planes Use connected component algorithm to find the regions. Pick top 4-5 regions. Consider the center each region as the normal you are looking for. Use connected component algorithm to find the regions. Pick top 4-5 regions. Consider the center each region as the normal you are looking for. Classify 3D points using the normals found above. Reclassify points that have the same normal to more clusters based on their distance from the origin in the direction of the normal.

Finding Planes

Filling Windows Find 4 corners for each plane. Project points onto the bounded plane Choose the resolution properly. Choose the resolution properly.

Filling Windows

Find the windows in the image Transform the new points back to 3D.

Some Results

Future Work Removing junk data behind the windows: Find the edges of windows in the orthographic image. Find the edges of windows in the orthographic image. Create a frustum with its apex at the scanner origin and its base being the boundaries of a window. Create a frustum with its apex at the scanner origin and its base being the boundaries of a window. Remove all the data that is inside the frustum and behind the window plane. Remove all the data that is inside the frustum and behind the window plane.

Future Work

The Interface Hough pick 3D Window fill

Thanks to Professor Wolberg Professor Stamos Siavash Zokai Gene Yu