3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008

Outline Problem Data Set MRF Noise Reduction Multi-view Reconstruction Conclusion

Problem Create broad-view high-resolution 3D view 3D View Normal Camera Depth Camera

Data Set

MRF Single view super-resolution reconstruction Objective function: E=Ed+Ec Ed: Similarity between the up-sampled depth and the depth sensor measurement Ed: Similarity between the up-sampled depth and the depth sensor measurement Ec: Regions with similar color have similar depth Ec: Regions with similar color have similar depth mrfDepthSmooth code from Stephen Gould [1] James Diebel, Sebastian Thrun, “An Application of Markov Random Fields to Range Sensing”, Proceedings of Conference on Neural Information Processing Systems (NIPS), MIT Press, Cambridge, MA, 2005.

MRF: Result Original depth map MRF result

Noise Reduction Single-view Improvement Median filtering Median filtering Occlusion boundary removal Occlusion boundary removal Original depth imageMedian filtered depth image

Original MRF Median filtered Occlusion boundary removed

Multi-view Reconstruction Problem: misalignment

Multi-view Reconstruction New Objective function using multi- view information: E=Ed+Ec+Em Em: similarity between depth in two different view

Multi-view Reconstruction

Multi-view Reconstruction: Result

Conclusion Median filter is effective in removing sensor noise Removing occlusion boundary reduce noise due to motion By using information from multi-view, depth images are better aligned