1. 1 Recent Advances in Real-Time Hyperspectral Image Processing Mingkai Hsueh Remote Sensing Signal and Image Processing Laboratory Department of Computer Science and Electrical Engineering University of Maryland Baltimore County 1000 Hilltop Circle, Baltimore, MD 21250

2. 2 Outline Introduction to Hyperspectral Image Processing and its Applications Introduction to Hyperspectral Image Processing and its Applications Anomaly Detection Anomaly Detection  Anomaly Detection  Real-time implementation  Speed-up of Adaptive Causal Anomaly Detection Conclusions Conclusions Projects Projects

3. 3 Water Mixed pixel (soil + mineral) Mixed pixel (trees + soil) Hyperspectral Image

4. 4 Applications of Hyperspectral Image Processing Applications Applications  Man-made objects: canvas, camouflage, military vehicles in defense applications  Toxic waste, oil spills in environmental monitoring  Landmines  Trafficking in law enforcement  Chemical/biological agent detection  Special species in agriculture, ecology

5. 5 Types of Signatures Endmembers: Endmembers: Pure signatures for a spectral class used for spectral unmixing Pure signatures for a spectral class used for spectral unmixing Anomalies: Anomalies: Signals/signatures spectrally distinct from Signals/signatures spectrally distinct from their surroundings, i.e., abnormality. their surroundings, i.e., abnormality.  rare minerals in geology  abnormal activities in military applications.

6. 6 RX Algorithm RX algorithm basically performs the Mahalanobis distance that is specified by RX algorithm basically performs the Mahalanobis distance that is specified by (r i -  ) T × (K) -1 × (r i -  ) The required mean vector μ hinder the possibility of implementing the algorithm in real-time fashion. The required mean vector μ hinder the possibility of implementing the algorithm in real-time fashion.

7. 7 Causal RX Filter (CRXF) By replacing the covariance matrix by correlation matrix, we can achieve the real-time processing. By replacing the covariance matrix by correlation matrix, we can achieve the real-time processing. The functional form of CRXF The functional form of CRXF r i T × (R i ) -1 × r i r i T × (R i ) -1 × r i The major drawback is that if a detected anomaly remains on the image to be processed, it may decrease the detectability of the following anomalies. The major drawback is that if a detected anomaly remains on the image to be processed, it may decrease the detectability of the following anomalies.

8. 8 Adaptive Causal Anomaly Detector (ACAD) ACAD has the same functional form as does CRXF, except the sample correlation matrix R’ is formed by all the arrived pixel vectors except the detected anomalous target pixel vectors that have been removed. ACAD has the same functional form as does CRXF, except the sample correlation matrix R’ is formed by all the arrived pixel vectors except the detected anomalous target pixel vectors that have been removed. r i T × (R’ i ) -1 × r i An anomalous target map is generated at the same time as the detection process takes place. An anomalous target map is generated at the same time as the detection process takes place.

9. 9 HYDICE Data HYDICE (Hyperspectral Digital Imagery Collection Experiment) HYDICE (Hyperspectral Digital Imagery Collection Experiment)  15 panels of five types with three different materials.  They are arranged into a matrix in such a way that each row represents 3 panels of the same type with three different sizes, 3m  3m, 2m  2m, 1m  1m. Each column represents 5 panels of different types with the same size. Original imageTarget masked image Anomaly

10. 10 CRXF Results row 8row 16row 24row 32 row 40row 48row 56row 64

11. 11 ACAD Results row 8row 16row 24row 32 row 40row 48row 56row 64

12. 12 ACAD Target Map row 8row 16row 24row 32 row 40row 48row 56 row 64

13. 13 ACAD Hardware Design R i = R i-1 + r i × r i T (R i ) -1 = (Q i × R i upper ) -1 = ( R i upper ) -1 × Q i T δ ACAD (r i ) = r i T × (R i T ) -1 × r i t K ≤ τ Auto Correlator QR Matrix Inverse Abundance Calculation Anomalous Target Discriminator

14. 14 (A+BCD) -1 = A -1 – A -1 B(C -1 +DA -1 B) -1 DA -1  (A+rr T ) -1 = A -1 – (A -1 rr T A -1 ) / (1+r T A -1 r) By Woodbury’s identity, set B a column vector, C a scalar of unity, and D a row vector Let A be the current correlation matrix and r be the incoming pixel vector. Let A be the current correlation matrix and r be the incoming pixel vector. Matrix Inversion Lemma

15. 15 Matrix Inversion Lemma (Cont’d) With Matrix Inversion Lemma (MIL), we only need to compute With Matrix Inversion Lemma (MIL), we only need to compute Using MIL the matrix inversion is reduced to matrix multiplications. Using MIL the matrix inversion is reduced to matrix multiplications. Simulation is provided to evaluate the performance of MIL. Simulation is provided to evaluate the performance of MIL.

16. 16 ACAD Hardware Design R i = R i-1 + r i × r i T (R i ) -1 = (Q i × R i upper ) -1 = ( R i upper ) -1 × Q i T δ ACAD (r i ) = r i T × (R i T ) -1 × r i t K ≤ τ Auto Correlator QR Matrix Inverse Matrix Inversion Lemma Abundance Calculation Anomalous Target Discriminator

17. 17 Speed-up of MIL We use two versions of the MATLAB program to perform the ACAD on the same image cube. One uses the MATLAB inv() function and another one uses the MIL. We use two versions of the MATLAB program to perform the ACAD on the same image cube. One uses the MATLAB inv() function and another one uses the MIL. As we can see, the speed-up is about “2” times faster for the 64x64 HYDICE image than the one without MIL. As we can see, the speed-up is about “2” times faster for the 64x64 HYDICE image than the one without MIL. With MIL Without MIL Computation time 26.609045.6560

18. 18 The Matrix Inversion Lemma has been successfully applied to reduce the matrix inversion performed by Adaptive Causal Anomaly Detection (ACAD) into matrix multiplications. The Matrix Inversion Lemma has been successfully applied to reduce the matrix inversion performed by Adaptive Causal Anomaly Detection (ACAD) into matrix multiplications. Since the Causal RX Filter (CRXF) and Real-time CEM (Constrained Energy Minimization) previously proposed in Wang [2003] also involve inverse matrix computation, the same MIL-based approach can be also applied to reduce the computational load. Since the Causal RX Filter (CRXF) and Real-time CEM (Constrained Energy Minimization) previously proposed in Wang [2003] also involve inverse matrix computation, the same MIL-based approach can be also applied to reduce the computational load. Conclusions

19. 19 Future Work An effective Dimensionality Reduction (DR) or Band Selection (BS) may need to reduce the number of bands to an acceptable range so that we can further reduce the computation cost in both applications. An effective Dimensionality Reduction (DR) or Band Selection (BS) may need to reduce the number of bands to an acceptable range so that we can further reduce the computation cost in both applications. Heterogeneous platform may be also considered to reduce the design time and possibly achieve better performance. Heterogeneous platform may be also considered to reduce the design time and possibly achieve better performance.

20. 20 Projects Conducted in RSSIPL Joint Service Agent Water Monitor Joint Service Agent Water Monitor  Mission  Develop GUI image analysis software for detecting Biological Threat Agent on Handheld Assays  Ported developed algorithms onto embedded system, Stargate Gateway (SPB400, Linux single board computer) with external hand held scanner device.  Sponsor  US Army Edgewood Chemical and Biological Center (ECBC)  ANP Technologies, Inc.

21. 21 Projects Conducted in RSSIPL (Cont’d)

22. 22 Projects Conducted in RSSIPL (Cont’d) Multi-band Multi-threat warning sensor Multi-band Multi-threat warning sensor  Mission  Developed detection algorithms for missile and grenade images captured from real-time Multispectral imaging system.  Developed MATLAB based GUI for image analysis.  Sponsor  Surface Optics Corporation (SOC)

23. 23 Software for Detecting Agents

24. 24 Embedded System

25. 25 Projects Conducted in RSSIPL (Cont’d) Multi-band Multi-threat warning sensor Multi-band Multi-threat warning sensor  Mission  Developed detection algorithms for missile and grenade images captured from real-time Multispectral imaging system.  Developed MATLAB based GUI for image analysis.  Sponsor  Surface Optics Corporation (SOC)

26. 26 Publication Book Chapter Book Chapter  J. Wang, M. Hsueh and C.-I Chang, “FPGA Design for Second-order Statistics Based Target Detection Algorithm for Hyperspectral Imagery Applications,” High Performance Computing in Remote Sensing, Chapman & Hall/CR, Oct 2007.  J. Wang, M. Hsueh and C.-I Chang, “FPGA Implementation for Real-time Orthogonal Subspace Projection for Hyperspectral Imagery Applications,” High Performance Computing in Remote Sensing, Chapman & Hall/CR, Oct 2007.

27. 27 Publication (cont’d) Journal Journal  C.-I Chang and M. Hsueh, “Characterization of Anomaly Detection in Hyperspectral Imagery,” Sensor Review, Volume 26, Issue 2, pp. 137-146, 2006.  M. Hsueh and C.-I Chang, “Field Programmable Gate Arrays for Pixel Purity Index Using Blocks of Skewers for Endmember Extraction in Hyperspectral Imagery,” International Journal of High Performance Computing Applications, Dec 2007. (to appear)  C.-I Chang, M. Hsueh, F. Chaudhry, W. Liu, C.-C. Wu, G. Solyar, “A pyramid-based block of skewers for pixel purity index for endmember Extraction in hyperspectral imagery,” International Journal of High Speed Electronics and Systems. (to appear)  M. Hsueh and C.-I Chang, “Adaptive Causal Anomaly Detection on Reconfigurable Computing,” IEEE Transaction on Industrial Electronics. (To be submitted)

28. 28 Publication (cont’d) Conference Conference  M. Hsueh and C.-I Chang, “FPGA implementation of Adaptive Causal Anomaly Detection,” 2006 CIE Annual Convention, Newark, NJ, Sep 16, 2006.  C.-I Chang, M. Hsueh, F. Chaudhry, W. Liu, C. C. Wu, A. Plaza and G. Solyar, “A Pyramid-based Block of Skewers for Pixel Purity Index for Endmember Extraction in Hyperspectral Imagery,” 2006 International Symposium on Spectral Sensing Research, Bar Harbor, ME, May 29 to Jun 2, 2006.  D. Valencia, A. Plaza, M. A. Vega-Rodriguez, R. M. Perez and M. Hsueh, “FPGA Design and Implementation of a Fast Pixel Purity Index Algorithm for Endmember Extraction in Hyperspectral Imagery,” SPIE Optics East, Boston, MA, Oct 23-26 2005.  L. Wu, J. Wang, B. Ramakrishna, M. Hsueh, J. Liu, Q. Wu, C. Wu, M. Cao, C. Chang, J. L. Jensen, J. O. Jensen, H. Knapp, R. Daniel, R. Yin, “An embedded system developed for hand held assay used in water monitoring,” SPIE Optics East, Boston, MA, Oct 23-26, 2005.

29. 29 Publication (cont’d) Conference Conference  M. Hsueh and C.-I Chang, “Adaptive Causal Anomaly Detection for Hyperspectral Imagery”, IEEE International Geoscience and Remote Sensing Symposium, Alaska, Sep 19-26, 2004.  M. Hseuh, A. Plaza, J. Wang, S. Wang, W. Liu, C.-I Chang, J. L. Jensen and J. O. Jensen, “Morphological algorithms for processing tickets by hand held assay,” OpticsEast, Chemical and Biological Standoff Detection II (OE120), Vol. 5584, Philadelphia, PA, Oct 25-28, 2004.  C.-I Chang, H. Ren, M. Hsueh, F. D’Amico and J.O. Jensen, “A Revisit to Target-Constrained Interference-Minimized Filter,” 48th Annual Meeting, SPIE International Symposium on Optical science and Technology, Imaging Spectrometry IX ( AM110), San Diego, CA, Aug 3-8, 2003.  S. T. Sheu, M. Hsueh, “An Intelligent Cell Checking Policy for Promoting Data Transfer Performance in Wireless ATM Networks,” IEEE ATM Workshop '99, Kochi City, Kochi, Japan, May 24-27, 1999.

30. 30 Thank you!!