Presentation is loading. Please wait.

Presentation is loading. Please wait.

Demosaicking for Multispectral Filter Array (MSFA)

Similar presentations


Presentation on theme: "Demosaicking for Multispectral Filter Array (MSFA)"— Presentation transcript:

1 Demosaicking for Multispectral Filter Array (MSFA)
Lidan Miao AICIP Research August 24, 2004

2 Background Challenge in acquisition of multispectral images.
The CyberEye 2100 is developed in SCP Laboratory at Syracuse University. A synchronized filter wheel and optical system provides 12 independent spectral bands from nm . Multispectral filter array technique. System Structure Camera Len MSFA Sensor Mosaic Image Reconstruction Scene Multispectral Camera_System/Color_Filter_Array_01.htm

3 Terminology MSFA samples Mosaic Image Demosaicking & Interpolation
Reconstructed image & Demosaicked image

4 CFA Demosaicking Review
Bilinear Interpolation Constant Hue-based Interpolation Hue is defined as the ratio or difference of red and green, blue and green (i.e. R-G,B-G or R/G, B/G) The HVS is more sensitive to hue artifacts than luminance errors. Gradient Based Interpolation Interpolate along the edges instead of across them. Weighted sum Interpolation

5 MSFA Review Design requirement A generic MSFA generation algorithm
Probability of appearance(POA) Spectral consistency Uniform distribution A generic MSFA generation algorithm Based on the binary tree decomposition. Input: the number of spectral band and the probability of each band.

6 MSFA Review Resulted MSFA 1 3 2 4 5 6 Decomposition Subsampling 7 8 1

7 Spectral Correlation Why??
Details are well preserved in spectral bands with more MSFA samples. Different spectral bands possess similar edge information. Validation metric: image similarity.

8 Spectral correlation:example
Similarity between different spectral bands B1 B2 B3 B4 B5 B6 B7 1 0.68 0.12 0.35 0.37 0.42 0.44 0.45 0.40 0.34 0.32 0.84 0.76 0.72 0.90 0.85 0.94

9 Progressive Demosaicking
Binary tree MSFA Band selection Pixel selection Interpolation Pixel distribution of each band

10 Band Selection Determine the order of selecting different spectral bands for interpolation. Band selection is associated with the tree level of each leaf, i.e. the probability of appearance. The spectral band with the highest POA is interpolated first.

11 Pixel Selection Binary tree traversal from leaf nodes
Example: reconstruct spectral band “C”

12 Interpolation Gradient Based Interpolation
Pros. vs. Cons. Adaptive edge-sensing Interpolation: weighted sum of neighboring pixels. p3 p1 e p2 p4 p1 p3 e p4 p2

13 Edge-sensing Interpolation
Edge-likelihood in each direction of neighboring pixels. Sobel edge detector and the second order derivative. Weight estimator Estimation of target pixel value

14 Case studies POA=1/2 POA=1/4 POA=1/8

15 Experiments-Data Description
Real Multispectral Data 92AV3C9(9 bands) FLC1(12 bands) TIPJUL(7 bands) Synthetic Multispectral Data Eight objects

16 Validation Original multispectral images are sampled using MSFA to generate mosaic images. Reconstruct the mosaic images using proposed method with and without binary tree bilinear interpolation with and without binary tree. Compare between the original multispectral images and the reconstructed images. Evaluation metric: subject visual comparison, objective RMSE, classification accuracy.

17 Visual Comparison Proposed with binary tree
Proposed without binary tree Bilinear with binary tree Bilinear without binary tree Original

18 Visual Comparison Original Proposed with and without tree
Bilinear with and without tree

19 Experiments-RMSE Root Mean Square Error
where M and N are the size of image, and is the original image, represents the reconstructed image. The smaller the RMSE, the better the reconstructed image.

20 Experiments-RMSE Comparison
Real multispectral data set Synthetic multispectral data set

21 Experiments-Pixel Classification
Real multispectral data Each scene contains different classes. Choose the region we know the ground truth, from which we extract training and testing samples. Knn classifier. Synthetic multispectral data Extract 20 training samples from each targets, all the targets pixels are testing samples. Knn classifier

22 Pixel classification accuracy
Real multispectral data set Synthetic multispectral data set

23 Experiments-Color Map
Pixel classification of each targets, different colors represent different classification results Original Reconstructed Original Reconstructed Original Reconstructed

24 Experiments-Object classification
Compare the spatial and spectral features. Spatial: hu-moments (7D) Spectral: average of target pixels (7D) Study the effect of viewing distance. A schematic representation for extraction of training and testing set

25 Object classification accuracy
Original Proposed Bilinear

26 Conclusion The proposed progressive method outperforms non-progressive and bilinear interpolation. The classification accuracy using reconstructed images is comparable with the one generated using original multispectral images. The MSFA technique is a feasible solution for multispectral cameras.

27 Thanks ?


Download ppt "Demosaicking for Multispectral Filter Array (MSFA)"

Similar presentations


Ads by Google