1. Preprocessing for Hyperspectral Analysis
Calibration, Noise, Bad Bands, Atmosphere

2. AVIRIS Reflectance Scene: Cuprite, Nevada

3. Overview
Hyperspectral data require extensive preprocessing so that they can produce useable spectral information
Typically convert to ground reflectance so that data can be compared to spectral libraries
Calibration (conversion of DNs to radiance)
Atmospheric correction
Often need to remove “bad bands” and create spectral subsets
May need to identify pixels that represent spectral endmembers for unmixing

4. Goal is to compare sensor-derived spectral curves to spectral libraries or ground measurements.
From Map and Image Processing System (MIPS) online user’s manual

5. Identification and flagging of bad bands
Not uncommonly, many bands in a hyperspectral dataset can be unusable
Low (poor) SNR
No or low radiance due to atmospheric absorption (causes low SNR because signal is low)
Sensor problems

6. AVIRIS Band 3 from Cuprite, Nevada reflectance image

7. AVIRIS Band 112 from Cuprite, Nevada reflectance image

8. Atmospheric transmittance across spectrum
Band 112
From Map and Image Processing System (MIPS) User manual (online)

9. Erdas tool for flagging bad bands in a dataset

10. Choosing spectral subsets
Accuracy of classifications can actually decrease with increasing number of bands
Sometimes called the Hughes Phenomenon or the “curse of dimensionality”
High dimensional statistics require more input data (e.g., training pixels) to work effectively
Many bands in a hyperspectral dataset are redundant (correlated) and therefore not needed.
Many bands may be irrelevant for certain targets or applications.

11. Erdas tool for omitting unwanted bands from an analysis

12. Which areas might you omit (or choose) if you need to ID montmorillonite?

13. Techniques for choosing spectral subsets.
Removal of atmospheric absorption regions (often “bad bands”)
Awareness of key spectral features of target(s)
Principal Components Analysis (PCA)
Can be used to reduce dimensionality of hyperspectral data IF you are not interested in the actual spectral reflectance curves
Lower PCs contain most of the information in a dataset.
PCA can also help identify endmembers for spectral unmixing.
Can be problematic when amount of noise varies from band to band in an image
Minimum Noise Fraction (MNF) – a variation of PCA that adjusts for bands that have uneven amounts of noise.

14. Minimum Noise Fraction (MNF)
When noise varies from band to band, PCA produces a set of components that don’t have increasing noise with increasing component # (PC1, PC2, PC3…PC224).
It is often desirable to cloister the noise in the higher order components (PCs)
MNF Analysis is a modified version of PCA that creates new “bands” (components) with increasing noise content as you go up through the components
Can invert to re-create bands that have a Gaussian noise distribution.
Concentrates the information in low order components and noise in high order components.

15. Erdas MNF tool.

16. Calibration
Calibration is the process of converting raw DNs to radiance.
Requires internal calibration information for each sensor
Often improved by on-the-ground measurement of known targets

17. Uncalibrated at-sensor brightness: combination of atmospheric effects, sensor characteristics, and reflectance of the lake bed target (red square)
From Map and Image Processing System (MIPS) User manual (online)

18. Spectral curve from lakebed in same AVIRIS image after conversion to reflectance

19. Atmospheric correction (Conversion of at-satellite radiance to ground reflectance)
Hyperspectral analysis usually requires correction for both scattering (path radiance) and transmittance
Often use radiative transfer models (based on the physics of the interaction of light with atmospheric components)
Can be accomplished with ground measurements or image-based techniques
e.g., flat field correction, empirical line method

20. Atmospheric correction programs are available "off the shelf" or modified
MODTRAN: MODerate resolution atmospheric TRANsmission – models transmission of light through the atmosphere
6S: Second Simulation of a Satellite Signal in the Solar Spectrum
ACORN: Atmospheric CORrection Now
ATREM: ATmospheric REMoval (modeled after MODTRAN)
FLAASH: Fast Line-of-site Atmospheric Analysis of Spectral Hypercubes
ATCOR: ATmospheric CORrection
HATCH
Others…

21. ATREM removal of atmospheric interferance from satellite radiance
AVIRIS data: Kansas City
Water vapor image “removed” by ATREM

22. Kansas City image

23. Atmospheric transmittance
Strongly affected by several atmospheric constituents
Water vapor
Carbon dioxide
Ozone
Nitrous oxide
Carbon monoxide
Methane
Oxygen
Also affected by amount of aerosol in atmosphere (larger particles)

25. How do these programs work?
Two basic strategies
Use field data (e.g., from a radiosonde) to model atmospheric conditions in the “column” between ground and top of atmosphere (e.g., MODTRAN)
Infer atmospheric conditions from key hyperspectral bands that correspond to atmospheric absorption dips (e.g. ATREM)

26. Example: MODTRAN4 User’s Manual (current version is 5)
More info available at MODTRAN website:
Note that MODTRAN is complicated (manual is 99 pages long) and requires a lot of investment just to learn to run. Some companies have created more user-friendly front end interfaces for it.

27. Standard MODTRAN atmospheres
Default atmospheric profiles include seasonal representations of:
Atmospheric pressure
Temperature
Density
Water vapor
Ozone

28. Other MODTRAN parameters include
Scattering options
Geometry
Amounts of atmospheric constituents
Solar irradiance
Sensor characteristics
Etc.

29. Example: ATREM calibration of AVIRIS data in Park City, Utah
USGS Spectroscopy Lab used ATREM to correct AVIRIS data using a fairly uniform dam site
Data correction programs like ATREM don’t do a perfect job and need to be calibrated (fudged) for particular images
Data calibration was for a project characterizing abandoned mine lands (AML) in Utah

30. Simplified schematic of procedure
AVIRIS data
ATREM
model
Per pixel ground reflectance

31. Procedure Choose a calibration site on the ground
Should be large (multiple pixels)
Should be spectrally uniform
Should be spectrally neutral (not extremely bright or dark)
Collect spectral data on the ground using a handheld spectral device (e.g., ASD spectroradiometer)

32. Calibration Site: Deer Creek, Utah

33. Field spectra collection using Analytical Spectral Devices (ASD) radiometer

35. Procedure (cont.) Run the ATREM model on the AVIRIS image
Translate ground spectral bands to match the spectral band position and width of corrected AVIRIS data
Correct for path radiance using dark pixels from another part of image (ATREM tends to overcorrect for path radiance)
Dark pixels chosen for path radiance correction

36. Before calibration!

37. Procedure (cont.)
Dark pixel spectrum is used to create an “offset spectrum” that will be subtracted from ATREM corrected AVIRIS data (essentially dark pixel subtraction)

38. Procedure (cont.)
Extract field calibration site pixels from corrected AVIRIS data

39. Procedure (cont.)
Divide field spectrum by ATREM-corrected spectrum to create a “multiplier spectrum” which can be multiplied by every AVIRIS pixel spectrum to calibrate.

40. Procedure (cont.)
Apply offset and multiplier spectra to ATREM corrected imagery
Applied to all pixels, not just the ones on the calibration site
Allows corrected image to more closely match what would be measured if you were on the ground

41. Other atmospheric correction strategies
Empirical line method (image based)
Measure reflectance on ground at two sites with relatively high and low brightness, but which also have relatively constant reflectance across bands
Get satellite radiance for the two sites and graph points
Use slope and intercept of the line connecting the two to correct data

42. Other methods (cont.) Flat field correction
Find an area in the image that has relatively constant (“flat”) and high reflectance (to minimize noise) across the bands you are correcting.
Extract spectral reflectance of flat field from image
Divide all pixels in all bands by the corresponding flat field values
Gives relative reflectance (not absolute)
Can be difficult to find a flat field
Dry lakebeds work well.
Spectral features (where it isn’t spectrally “flat”) in the flat field will cause problems

43. Summary
Image pre-processing for hyperspectral analysis is a rigorous and time-consuming task that needs to be done well or projects will not be successful. Worth investing considerable time into pre-processing.