1. Reconstructing 3D mesh from video image sequences supervisor : Mgr. Martin Samuelčik by Martin Bujňák specifications Master thesis MartinBujnak@post.sk http://www.2B3D.host.skMartinBujnak@post.skhttp://www.2B3D.host.sk

2. contents… aims requirements analysis software design –algorithms selection (criteria) –data description, I/O characteristics (assumptions) –dependencies / independencies (HW/SW) –function specification –user model –GUI Martin Bujňák : Reconstructing 3D mesh from video image sequences. MartinBujnak@post.sk http://www.2B3D.host.skMartinBujnak@post.skhttp://www.2B3D.host.sk

3. Martin Bujňák : Reconstructing 3D mesh from video image sequences. aims… reconstruction of 3D mesh defined by continuous sequences of images, captured by standard (middle class) hand-held video camera reconstruction should be fully automatic or with very little user interactions output should by textured 3D model MartinBujnak@post.sk http://www.2B3D.host.skMartinBujnak@post.skhttp://www.2B3D.host.sk

4. requirements analysis… find corresponding feature points in input images – feature detectors (to slow) – tracking (assumptions on input) enumerate relative transformation between spaces defined by neighboring input images camera trajectory reconstruction error correction (3D snapping in 2D) mesh and textures reconstruction… Martin Bujňák : Reconstructing 3D mesh from video image sequences. MartinBujnak@post.sk http://www.2B3D.host.skMartinBujnak@post.skhttp://www.2B3D.host.sk

5. image correspondence… difficult process - big cpu/time consumption continuous image motion used as hint for correspondence problem –brute-force comparison –but nearest in motion direction first problems –specular highlights (in feature tracking problem, in surface extraction helper) homomorphous filters (garage filter) –bad features requirements analysis Martin Bujňák : Reconstructing 3D mesh from video image sequences. MartinBujnak@post.sk http://www.2B3D.host.skMartinBujnak@post.skhttp://www.2B3D.host.sk

6. relative transformation assuming that at least 5 well detected corresponding feature points exist if not skip image epipolar geometry for computing projection matrices camera calibration –enumerated from input –2 degree polynomial suffice to fix camera lens deformation –focal point from image correspondence improving accuracy –all camera positions lay on curve –projection matrix basis vector are interpolated continuously requirements analysis Martin Bujňák : Reconstructing 3D mesh from video image sequences.

7. mesh and textures reconstruction… direct volume reconstruction –intersections in image space using normalized colors problems –numerical stability requirements analysis Martin Bujňák : Reconstructing 3D mesh from video image sequences.

8. software design… pipe concept… Martin Bujňák : Reconstructing 3D mesh from video image sequences. image i edges detector homomorphous filter + expected value trash holding feature extraction previous results in fine structure motion estimation image i-j filter relative camera transformation error correction (cam. trajectory snapping) surface + textures extraction correspondence happy user ;) error correction

9. main criteria is speed edges detected using differences edge reconstructed using curves (polynomial of low degree) feature point = intersection of more curves (in some epsilon neighborhood) openCV (static) or own (dynamic) software design algorithms selection … Martin Bujňák : Reconstructing 3D mesh from video image sequences.

10. not image-based (cross-correlation / invariant regions) correspondence technique –slow –inaccurate statistical image precision –previous images not used many information ignored… tracking technique – combined with brute-force matching… software design algorithms selection … Martin Bujňák : Reconstructing 3D mesh from video image sequences.

11. surface reconstruction - similar to silhouette based techniques (on edged) –clipping (sweep & prune) extruded edge curves –requires robust algorithm (booleans) –still searching for it non-silhouette parts (inner points) –intersect all extruded cones from every pixel (the same problems as meant above) –idea : restriction planes each image restricts move of its point – leads to big equation point cloud or displacement map… software design algorithms selection … Martin Bujňák : Reconstructing 3D mesh from video image sequences.

12. input – video file, of video stream –handled by operating system and codecs –internally every frame will be 24bit RGB 888 bitmap –float precision required in specular highlights removal algorithms (homomorphous operation -> DCT -> complex numbers) –assuming “continuous” input (not time-step) software design data description, I/O characteristics... Martin Bujňák : Reconstructing 3D mesh from video image sequences.

13. output – mesh and textures –mesh will be exported by supporting 3D environment (go3D) – minor control (but wrml, 3ds, scn…) –textures in some standardized image formats supported by go3D (bmp, tga, jpeg…) internal data structures –topological maps, 3D hashed grid, … software design data description, I/O characteristics... Martin Bujňák : Reconstructing 3D mesh from video image sequences.

14. will be implemented as plug-in into modeling 3D environment plug-in –OS dependent interface to video devices (direct show) –GUI using environment framework OS independent –C++ open CV software design dependencies / independencies (HW/SW)... Martin Bujňák : Reconstructing 3D mesh from video image sequences.

15. selecting input device process start exporting output subject of change software design function specification… Martin Bujňák : Reconstructing 3D mesh from video image sequences. aim : user not needed user model… every function have its own button in some layout GUI…