1. CLOUD DETECTION WITH OPERATIONAL IMAGERS
AOS LECTURE
CLOUD DETECTION WITH OPERATIONAL IMAGERS
OUTLINE:
What is a Cloud Mask
Review of Cloud Mask Radiative Transfer
Properties of the Operational Imagers (AVHRR and GOES)
Contrast Tests
Spatial Tests
Temporal Tests
Spectral Tests
R2/R1, Split window, near – far window, 4 micron ref.
Discriminating Cloud Types

2. Who Am I? I work for the Office of Research and Applications with NOAA
My group develops algorithms (including cloud masks), radiative transfer techniques and calibrates the data from the NOAA Operational Sensors
My Responsibilities include some of the following
AVHRR cloud properties (including cloud mask)
GOES surface and some cloud properties (including cloud mask)
VIIRS (the next AVHRR) cloud algorithms

3. What is a cloud mask More than 50% of the globe is cloudy.
Almost all algorithms need pixels that are either or cloudy – few can handle cloud contamination
For some parameters – SST or Vegetation Indices, cloud masking can be the number one source of error.
Cloud masking is very much “in the eye of beholder” and one mask rarely pleases everyone
The radiative transfer knowledge needed for cloud masking requires knowledge of many features of atmosphere, surface and cloud radiative properties.
Cloud masking is probably the most difficult thing to do right in imager remote sensing.

4. Example Cloud Mask from AVHRR
Cloud masks have to balance coverage and quality of clear data
0.63 micron reflectance
Cloud Mask + clear 11 micron temp.

5. Ways to Write a Cloud Mask
Automatic Classifiers (Neural Networks)
Develop a training data-set where you “manually” detect clear and cloudy pixels
Once trained, a classifier can be applied to cloud mask images
Reduces need to physically identify cloud tests
2. Sequential Decision Trees
This is most common and used in ISCCP, MODIS and CLAVR
Design a series of single tests that used in sequence
Results of one test can be used to effect subsequent tests
Traditionally thresholds used in tests were static
Now, radiative transfer modeling on the fly is becoming a feasible option for setting thresholds.

6. The Radiative Transfer Behind Cloud Masking
To know what a cloud is , you need to know what a cloud isn’t
You need to be able discriminate the effects of cloud from those of The atmosphere and the surface
Three Driving Effects on the what the satellite sees
the variation in the source – solar reflection or thermal emission
Variation in the atmosphere – scattering, transmission and emission
The surface – its emissivity and its temperature relative to the clouds
Variation in cloud signatures – reflectance and emissivity.
You have to to understand all three to be able to uniquely detect cloud.
Some regions have surface or atmospheric conditions (ie inversions) that complicate cloud detection

7. In case, you need to review your emission spectra….
Reflection of sunlight
Emission from earth

8. UV – NEAR IR Atmospheric Transmission Spectra (0.5 – 3 microns)

9. AVIRIS Image - Porto Nacional, Brazil 20-Aug-1995
224 Spectral Bands: mm
Pixel: 20m x 20m Scene: 10km x 10km

10. Near-IR – FAR IR Atmospheric Transmission Spectra (2 – 15 microns)

11. GOES Sounder data – Shows how Infrared Observations vary spectrally.X X X X X X

12. The Surface also presents a variation of reflectance/emissivity
Note the chlorophyll behavior from 0.6 to 0.8 microns
In the infrared, most surfaces are blackbodies except
deserts (especially at 4 micron)

13. Clouds offer another spectral variation that has to be accounted for …
Extinction efficiency is related to optical depth and is therefore related to the reflectance and emissivity of a cloud.
As extinction efficiency gets smaller, the emissivity decreases (10-12 microns)

14. Operational Imagers:
In this lecture we are dealing with cloud masking applied to the operational imagers.
Operational – they are used for real-time products and if they blow up, they are replaced with an identical version.
Imagers are originally for taking images of weather patterns. They have:
high spatial resolution
few channels with wide responses
often poor calibration (but this can be fixed)
Meteorological imagers include GOES imager, AVHRR (we’ll discuss those) but also
MODIS – 36 channels, 1 km spatial resolution
ABI – the next GOES imager – 12 channels
SEVRI – Europe's Geo imager – 12 channels.
Landsat – 5 channels – 28.5 meter resolution – limited coverage

15. The Operational Imagers
1. The AVHRR – A Polar Orbitting imager
Designed in the 1970’s for non-quantitative cloud imagery
Can be calibrated well enough for other remote sensing apps.
5 channels with relative broad response functions
CH um
CH4 – 11 um
CH5 – 12 um
CH1 – 0.63 um
CH2 – 0.86 um

16. This is what AVHRR looks like…
0.6 um um um 11 um 12 um
NIGHT – NO REFLECTANCE

17. 2. The GOES Imager
5 channels like the AVHRR
Replaces 0.86 micron channels with a “water vapor channel” at
6.7 microns
4 km resolution at the equator
New GOES imagers replace 12 micron with 13.3 um co2 channel

18. Difference in data from Polar and Geostationary Orbitters
AVHRR
GOES

19. Geostationary Platforms Facilitates use of temporal tests

20. What is the difference between an imager and sounder?
1. While Sounders offer more channels, their spatial resolution is much worse

21. Sounder have more channels – 10’s (operational) or 100’s (research)

22. Cloud Mask Tests Types
All Cloud Masks uses four types of tests to look for cloud
Contrast tests – Look for contrast between a pixel and what we think a clear pixel should look like. (ie. Clouds are colder and brighter so anything really bright or really cold must be a cloud)
Spectral Tests – use knowledge radiative transfer to identify channel behaviours that can only mean a pixel is cloudy
Temporal Tests – look for rapid changes in a pixel that are greater than that possible for a clear pixel (if pixel drops by 20K in 1 hr – it is probably cloudy)
Spatial – Clouds are texturally different than the clear sky. At 1 – 4km resolution, the surface is usually more spatially uniform in temperature and reflectance than clouds. At resolutions less than 100 m – the surface can have more texture than clouds!
NO ONE TEST TYPE IS VALID ALL THE TIME, BUT SPATIAL AND TEMPORAL COME CLOSE

23. Examples of Using Contrast in Cloud Masking
Actually, all tests are contrast tests, but contrast tests usually mean
The temperature (11 micron) compared to clear sky
The visible reflectance compared to clear sky
Some regions prevent the use of both types of contrast test – cold snow.
Being able to stare at the same pixel from the same location, gives GOES an advantage in using contrast tests for cloud masking. The next images show a GOES visible image compared to visible composite made by storing the darkest pixel over 28 days.
Contrast tests are never enough for an accurate cloud mask

26. AVHRR Contrast Test Example – Comparison of SST versus Climatology
You can’t stare at the same spot from a polar orbitter. But Sea surface temperature does not vary much. Good basis for contrast test for a polar orbiter
SST from Climate
SST from AVHRR
Cloud Test Result

27. Contrast Test contd.
Where there is no reliable contrast, these test will fail!
(Example in the Poles were there is no contrast – thermal or visible)
Cloud warmer than
ice
Ice

28. Spatial Test Basis.
Assumption: Clear regions are more uniform than cloudy regions.
Very good robust test, regions were there is a lot of surface non-uniformity can cause problems. ( i.e. Mountains, sea-ice)
The “Golden Arches”
Plot temperature versus the local temperature standard deviation.
If there is only one cloud layer, you should get an arch like this.
Partially cloudy pixels have high variability
Clear foot
Cloud foot

29. Spatial Uniformity Example
The CLAVR cloud mask uses the max – min on a 2x2 pixel array as the estimate of the local variability.
Vary good at picking out partial or small scale cloudiness missed by others
Needs to be a function of surface type
11 micron temp.
Variability in 11 micron temp.
clear
LAND

30. Temporal Tests
Geostationary satellites offer high temporal resolution
Look for pixels that get brighter or colder from one image to the next
Not a cloud mask – just a sequence of ch1 images from GOES

31. Temporal Tests with Polar Orbiters
Polar Orbiters only see the same spot at the same time every few days so temporal tests are hard to implement (ie CLAVR or MODIS do not use them).
But at the poles, imagers like the AVHRR can use temporal tests like GOES does.
Here is an example of an animation of MODIS 11 micron temperature used to derive winds

32. Spectral Tests
A spectral test – looking a some channel or channel combination that definitively says this is cloud.
With only 5-6 channels, all AVHRR or GOES cloud masks tend to use the same spectral tests.
Reflectance Ratio of to 0.63 micron reflectance (AVHRR only)
Split Window (11 – 12 micron ) Temperature difference
Near – Far window (4 – 12 micron) Temperature Difference
Derived 4 micron reflectance /emissivity
Water Vapor – window temperature difference (GOES only)
Co2 channel tests (GOES only)
Even MODIS with 36 channels – these tests are the main body of the MODIS cloud mask

33. To illustrate these tests, we are going to show their application in three regions
#1 CARIBBEAN
#3 Nighttime Artic
#2 – Sahara / Jungle

34. Reflectance Ratio Test Basis
Based on our knowledge of reflectance spectra, we can predict:
R2/R1 = 1.0 for cloud (if you can’t see the surface underneath)
R2/R1 > 1.0 for vegetation ( look at pinewoods spectra)
R2/R1 << 1.0 for water
R2/R1 about 1 for desert
cloud
Glint is a big limiting factor
To this test over oceans.
Also, smoke or dust can look
Like cloud in R2/R1. R1 R2

35. Reflectance Ratio Test for Scene #1
R2/R1
R2 (0.86 micron)
R2/R1 < 1.0 over ocean
R2/R1 =
for cloud
R2/R1 > 1.0 over
the jungle
Cloud Test:
R2/R1 > 0.7 for water
R2/R1 < 1.1 over land

36. Reflectance Ratio Test Over Scene #2 – Saharan Desert / Tropical Africa
R2/R1 near 1
no contrast
with clouds
R2/R1 detects cloud
Over jungle
R2/R1 does not work
Over deserts

37. Reflectance Ratio Test over the Poles
No sunlight for extended period of time
When present, sun angles are high anyway
snow does not offer much contrast with clouds in R2/R1.
Other Issues of this test
Thresholds should be a function of angle
Knowledge of vegetation really helps in defining tests

38. Split Window Spectral Test Basis
T11  e11 Tc + 11Ts
Tc = cloud temperature
Ts = Surface temperature
= cloud emissivity (1 = black body)
 = cloud transmission (1 = transparent)
T12  e12 Tc + 12 Ts
For Thick Clouds ;  = 0 so:
T11 – T12  (e11- e12) Tc
For Thin cold (high) clouds, the surface term dominates
T11 – T12  ( e 12) Ts

39. For Clouds, e11 < e12 so T11 – T12 < 0
Split Window Spectral Test Basis
For Clouds, e11 < e12 so T11 – T12 < 0
For cirrus, 11 < 12 so T11 – T12 > 0
water
ice
Bigger potential in water cloud, but ice clouds are cold and have bigger net effect

40. Split Window Results for Scene 1
Thick cloud
Small T11 – T12
Thin cirrus
Big T11-T12
Clear ocean
Moderate T11-T12
(h2o effect)
Hot land
tricky

41. Split Window Applied to the Desert Scene #2
Spectral variation in desert emissivity can be tricky
Surface
Feature

42. Split Window Applied to the Artic Scene #3
Little water vapor in poles – so no problem separating water vapor from cloud
Inversions can cause T11-T12 to behave oppositely
Magnitude of signal much smaller than in tropics.
Low cloud
cirrus

43. Far – Near IR window Spectral Test
During night, same qualitative behavior as split window but more sensitive
T4 – T12 < 0 for thick cloud
T4 – T12 > 0 for cirrus
During Day, solar energy impacts the 4 micron observations.
T4-T12 >> 0 for any cold cloud
At 4 microns, equation relating radiance to temperature is very strong. So any additional signal from the sun or from the surface causes a big jump in T4.
Over oceans during day, glint is a problem

44. Far – Near IR Results for Scene 1
Cold cloud
T4 – T12 > 20 K
Clear ocean
T4 –T12 < 8 K

45. Far – Near IR Results for Scene 2 (Sahara)
Surface
features

46. Far – Near IR Results for Scene 3 (Artic)
Possibly the best test in artic at night as data is not too noisy
Low Cloud
CIRRUS

47. 4 micron Reflectance Image
You can estimate thermal contribution at 4 micron, subtract it off and the rest can be treated like a 4 micron reflectance channel.
Very good test over the ocean.
Almost useless over deserts

48. Other Spectral Tests
Water Vapor Channel – You can’t see the surface or lowlevel clouds. Useful in the artic (for MODIS) for detecting inversions. If you see an inversion, you aren’t looking at a cloud.
CO2 Channel – same as water vapor. Able to detect high cloud only.
Since high cloud is usually cold and/or bright, these tests do not add that much.

49. What’s New in Cloud Masking….
The 1.38 micron channel – a reflectance channel sitting in a water vapor absorption band.

50. How does the Automated AVHRR cloud mask perform on these scenes?
Cloud contamination would show up as cold pixels
Cold temps = noise

51. Cloud Typing From AVHRR / GOES
In addition to detecting cloud, we also want to determine what kind of cloud it is.
Using the same knowledge of cloud spectra we used for cloud detection,
We can tell if a cloud is:
Made of ice particles or water droplets (ie phase)
Optically thick or optically thin
Something optically thin over a warmer cloud

52. Cloud Type Results from AVHRR for that scene over the Caribbean.
Cloud types: water , supercooled water, thick ice, cirrus, overlapped cirrus

53. How do we determine phase?
The spectral variation of the imaginary component of index of refraction and its difference between ice and water is basis of the phase detection
Higher the value, the greater the absorption
4 um
8.5um
11um
12um
ice
water

54. Based on imaginary index of refraction, ice should absorb more (scatter less)
At 4 um than water clouds
This is the case

55. How do we detect cirrus over lower cloud?
One way is to look for pixels with high T11-T12 and high 0.65 um reflectance. Plane parallel theory says this can’t be – so this one way to flag multilayer cloud.
Another way (Baum) is to look for pixels that fall between the clusters of the 0.65 and 1.6 um reflectance um is like 3.75 um – ice absorbs more than water.