1. My Research Experience Cheng Qian

2. Outline 3D Reconstruction Based on Range Images Color Engineering Thermal Image Restoration

3. 3D – Overview To reconstruct the geometry and texture of a scene in a virtual environment. --- 3D scanning Create an arbitrary view by interpolation Method 1 ： 2D color-image-based reconstruction

4. 3D – Overview Method 2 ： Range-image-based reconstruction Range image -----Depth Intensity image --Reflectance CCD Image--RGB

5. 3D – Overview Range Image Intensity Image CCD Image A digital model Geometric Structure Materials Texture

6. 3D – Overview Preprocessing Registration Mesh Texture Mapping Lighting Shading ….. System architecture Raw Data (3D coordinates, Intensity, RGB) Visual Information (Geometry, Texture) Knowledge from Object Recognition Modeling

7. 3D – Range Image Registration Image of a left view Images registered Image of a right view Objective

8. 3D – Range Image Registration Scheme Range image I1Range image I2 Description based on geometric features and their interrelationships …… Construct feature correspondence M and measure the Similarity S between the two images ------ Find the Ｍ maximizing S Feature Extraction: surface, curve, corner point…… Noise filtering, outlier removing ……

9. 3D – Range Image Registration Noise filtering, outlier removing Before After Polar window filtering, Pseudo-median filtering Isolated point filtering

10. 3D – Range Image Registration Feature extraction Surface: adaptive-shape window Curve, corners: edge evolution

11. 3D – Range Image Registration Descriptions of the geometric features Interrelationship contained in nested geometric features Related geometric features are nested

12. 3D – Range Image Registration Correspondence and similarity measure Virtual features

13. 3D – Range Image Registration Improvement of the registration results Before After Global optimization

14. 3D – Range Image Registration What was left: Texture

15. Color Engineering Proposed a method for measuring luminance distribution of indoor scenes using a digital camera rather than an expensive luminance meter. Proposed a novel radiometric model for CCD sensors and a color self-calibration algorithm based on this model. The objective of this project is to calibrate the color performance of 100 CCDs in a lightfield-rendering system for 3D scene reconstruction. CCD 1 Fake CCD 2 CCD2 Transformed to be To approximate

16. Color Engineering Calibrated a line CCD sensor with poor color performance. The radiometric correlation between r, g, b channels is considered.

17. Thermal Imaging Proposed a radiometric model for infrared cameras and developed relevant model reconstruction methods, which resulted in obtaining a very precise forward function for thermal image restoration. Regularization techniques, such as Tikhonov, Total Variation, and Lasso, were applied to the restoration procedure and their performances were compared. Thermal camera model

18. Thermal Imaging Thermal image restoration Original imageImage restored by Tikhonov regularization, Edges are strongly penalized Image restored by Discontinuity- Adaptive model regularization, Edges are adaptively penalized Noise is suppressed Convexity of energy function is well controlled. Image restored by Total Variation regularization, Edges are preserved

19. Thermal Imaging With the adjustment of the camera setting, the point spread function (PSF) of the camera system can be changed. Therefore we try to develop a semi-blind image restoration algorithm that can recover the original image and the PSF simultaneously. Iteration 1 (a) Original image Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5 Iteration 6 Iteration 7 Iteration 8 Iteration 9 Final restoration results Iteration 9 Final restoration results

20. Thanks