1. Reconstruction with Adaptive Feature-Specific Imaging Jun Ke 1 and Mark A. Neifeld 1,2 1 Department of Electrical and Computer Engineering, 2 College of Optical Sciences University of Arizona Frontiers in Optics 2007

2. Outline Frontiers in Optics 2007  Motivation for FSI and adaptation.  Adaptive FSI using PCA/Hadamard features.  Adaptive FSI in noise.  Conclusion.

3. Motivation - FSI Reconstruction with Feature-specific Imaging (FSI) : Frontiers in Optics 2007 FSI benefits:  Lower hardware complexity  Smaller equipment size/weight  Higher measurement SNR  High data acquisition rate  Lower operation bandwidth  Less power consumption Sequential architecture: Parallel architecture: LCD G (NxM) Reconstruction matrix G (NxM) object object reconstruction DMD Imaging optics light collection single detector feature projection vector

4. Motivation - Adaptation Frontiers in Optics 2007  Acquire feature measurements sequentially  Use acquired feature measurements and training data to adapt the next projection vector  The design of projection vector effects reconstruction quality.  Using Principal Component Analysis (PCA) projection as example Testing sample Training samples Projection axis 2 Static PCA Projection axis 1 Reconstruction Adaptive PCA Projection axis 2 Projection value Training samples for 2 nd projection vector Projection axis 1 Reconstruction

5. Frontiers in Optics 2007 Object estimate y i = f i T x Calculate f i+1 Reconstruction Object x Update A i to A i+1 according to y i Computational Optics Calculate f 1 R i+1 Calculate R 1 from A 1 Adaptive FSI (AFSI) – PCA: i: adaptive step index A i : i th training set K (i) : # of samples for A i+1  High diversity of training data helps adaptation PCA-Based AFSI Testing sample K (1) nearest samples Projection axis Testing sample K (1) nearest samples Selected samples According to 1 st feature According to 2 nd feature K (2) nearest samples Projection axis 2 Projection axis 1 R i : autocorrelation matrix of A i f i : dominate eigenvector of A i y i : feature value measured by f i

6. Object examples (32x32):  Reconstructed object:  RMSE:  Feature measurements: where,  is the total # of features PCA-Based AFSI Frontiers in Optics 2007  Number of training objects: 100,000  Number of testing objects: 60

7.  RMSE reduces using more features  RMSE reduces using AFSI compare to static FSI  Improvement is larger for high diversity data  RMSE improvement is 33% and 16% for high and low diversity training data, when M = 250. Frontiers in Optics 2007 AFSI – PCA: PCA-Based AFSI K (i) decreases iteration index i Reconstruction from static FSI (i = 100) Reconstruction from AFSI (i = 100)

8.  Projection vector’s implementation order is adapted. Frontiers in Optics 2007 AFSI – Hadamard: Hadamard-Based AFSI Selected samples K (1) nearest samples testing sample projection axis 1 K (1) nearest samples testing sample projection axis 2 K (2) nearest samples sample mean First 5 Hadamard basis ←Static FSI AFSI→ according to 1st feature according to 2nd feature sample mean projection axis 1  Sample mean for training set A i is  y j = f i T j = 1,…,M  max{y j } corresponds to the dominant Hadamard projection vector

9.  L : # of features in each adaptive step Frontiers in Optics 2007 : sample mean of A i f i : i th Hadamard vector for A i AFSI – Hadamard: Hadamard-Based AFSI K (1) nearest samples testing sample projection axis 1 sample mean projection axis 2 Selected samples according to 1st 2 features Object estimate y iL+j = f iL+j T x (j=1,…,L) Choose f iL+1 ~ f (i+1)L Reconstruction Object x Update A i to A i+1 according to y iL+j Computational Optics Choose f 1 ~f L <Ai><Ai> Sort Sort Hadamard basis vectors

10.  RMSE reduces in AFSI compared with static FSI  RMSE improvement is 32% and 18% for high and low diversity training data, when M = 500 and L = 10.  AFSI has smaller RMSE using small L when M is also small  AFSI has smaller RMSE using large L when M is also large Hadamard-Based AFSI Frontiers in Optics 2007 AFSI – Hadamard: K (i) decreases number of features M = Li L decreases L increases number of features M = Li Reconstruction from adaptive FSI Reconstruction from static FSI

11. Hadamard-Based AFSI – Noise Frontiers in Optics 2007 AFSI – Hadamard:  Hadmard projection is used because of its good reconstruction performance  Feature measurements are de-noised before used in adaptation  Wiener operator is used for object reconstruction  Auto-correlation matrix is updated in each adaptation step  T : integration time  σ 0 2 = 1  detector noise variance: σ 2 2 = σ 0 2 /T Object estimate y iL+j = f iL+j T x+n iL+j (j = 1,2,…L) Choose f iL+1 ~f (i+1)L Reconstruction Object x Update A i to A i+1 according to Computational Optics Choose f 1 ~f L from de- noising y iL+j Calculate R i for A i Sort Hadamard bases Sort

12. Frontiers in Optics 2007  RMSE in AFSI is smaller than in static FSI  RMSE is reduced further by modifying R x in each adaptation step  RMSE improvement is larger using small L when M is also small  RMSE is small using large L when M is also large Hadamard-Based AFSI – Noise High diversity training data; σ 0 2 = 1 K (i) decreases L decreases L increases High diversity training data; σ 0 2 = 1 AFSI – fixed R x AFSI – adapted R x Static FSI

13. T : integration time/per feature; M 0 : the number of features Total feature collection time = T × M 0  Reducing Measurement error  Losing adaptation advantage Hadamard-Based AFSI – Noise Frontiers in Optics 2007 High diversity training data; σ 0 2 = 1 Minimum total feature collection time Increasing T Trade-off

14. Conclusion Frontiers in Optics 2007 Noise free measurements:  PCA-based and Hadmard-based AFSI system are presented  AFSI system presents lower RMSE than static FSI system Noisy measurements:  Hadamard-based AFSI system in noise is presented  AFSI system presents smaller RMSE than static FSI system  There is a minimum total feature collection time to achieve a reconstruction quality requirement