ECMWF/EUMETSAT NWP-SAF Satellite data assimilation Training Course 1 to 4 July 2013

ECMWF/EUMETSAT NWP-SAF Satellite data assimilation Training Course The detection and assimilation of clouds in IR radiances

Task 1: Overlay clear and cloudy Tropical IASI spectra Duplicate Tropical profile ( tropic.nc > copy_1_tropic.nc) Analyse copy_1_tropic.nc change cloud parameters (2D) Edit / visualise IASI SPECTRUM (with tropic.nc ) Edit / drop IASI SPECTRUM (with copy_1_tropic.nc)

Task 1: Overlay clear and cloudy Tropical IASI spectra

Option 1: Detect and reject cloud contaminated observations Option 2: Explicitly estimate cloud parameters from the radiances within the data assimilation (simultaneously with T,Q,O3 etc..)

Cloud detection methods Simple window channel departure checks Co-located imager checks Pattern recognition algorithms Hybrid systems

Cloud detection methods Simple window channel departure checks Co-located imager checks Pattern recognition algorithms Hybrid systems

Window channels have the highest sensitivity to cloud

Clear population Cold departures indicating cloud contamination in OBS Observed radiance at 11 um minus radiance calculated from background in clear sky

Y obs – H(X clear ) > dT threshold

…results in a asymmetric population surviving the check …!

Cloud detection methods Simple window channel departure checks Co-located imager checks Pattern recognition algorithms Hybrid systems

AVHRR imager pixels IASI field of view

Homogenous clear Homogenous cloudy Mixed cloud scene

AVHRR standard deviation AVHRR mean radiance 220K 250K 280K 300K Scatter stdv versus mean Tb

AVHRR standard deviation AVHRR mean radiance 220K 250K 280K 300K Homogenous clear

AVHRR standard deviation AVHRR mean radiance 220K 250K 280K 300K Homogenous cloudy

AVHRR standard deviation AVHRR mean radiance 220K 250K 280K 300K Mixed scene

Cloud detection methods Simple window channel departure checks Co-located imager checks Pattern recognition algorithms Hybrid systems

Zoom in …

Clear spectrum Cloudy spectrum

Break point

Cloud at 200hPa Cloud at 500hPa Cloud at 400hPa Zero Cloud Break point depends on cloud height

observed-calculated (K) Vertically ranked channel index First filter and find the break point Observed spectrum minus clear sky computed spectrum

observed-calculated (K) Vertically ranked channel index First filter and find the break point Observed spectrum minus clear sky computed spectrum

observed-calculated (K) Vertically ranked channel index Break point Channels rejected Channels retained

CLOUD AIRS channel 226 at 13.5micron (peak about 600hPa) AIRS channel 787 at 11.0 micron (surface sensing window channel) temperature jacobian (K) pressure (hPa) unaffected channels assimilated contaminated channels rejected

Cloud detection methods Simple window channel departure checks Co-located imager checks Pattern recognition algorithms Hybrid systems

Why do we need Hybrid systems ? Y obs – H(X clear ) > dT But sometimes large errors in the background can lead to: False rejection of a good observation Missed rejection of a bad observation

Option 1: Screen the radiance data and reject cloud/rain contaminated observations Option 2: Explicitly estimate cloud parameters from the radiances within the data assimilation (simultaneously with T,Q,O3 etc..)

It looks very difficult …why would we try to do this ?

CLOUD AIRS channel 226 at 13.5micron (peak about 600hPa) AIRS channel 787 at 11.0 micron (surface sensing window channel) temperature jacobian (K) pressure (hPa) unaffected channels assimilated contaminated channels rejected We have to throw away lots of data !!!

Sensitive areas and cloud cover Location of sensitive regions Summer-2001 (no clouds) monthly mean high cloud cover monthly mean low cloud cover sensitivity surviving high cloud cover sensitivity surviving low cloud cover From McNally (2002) QJRMS 128

…so what is required to do this ?

The cost function J (X) model state observations background error covariance observation* error covariance observation operator (maps the model state to the observation space) If we wish to assimilate cloudy radiance observations …..

The cost function J (X) model state must include clouds (clw,cic,cf)

Two approaches to assimilate cloud affected infrared radiances Simplified system: very simple cloud representation currently limited to overcast scenes no information on clouds taken from model no back interaction with model via physics Advanced system: very complex cloud representation all cloud conditions treated information on clouds taken from model back interaction with model via physics X=(T,Q,V,ciw,clw,cc) X=(T,Q,V,cp,cf) cp cf

The cost function J (X) Background error covariance must include clouds (clw,cic,cf)

The cost function J (X) Obervations operator (RT and Model) must include clouds (clw,cic,cf)

Potential difficulties The cloud uncertainty in radiance terms may be an order of magnitude larger than the T and Q signal (i.e. 10s of kelvin compared to 0.1s of kelvin) The radiance response to cloud changes is highly non- linear (i.e. H(x) = H x (x)) Errors in background cloud parameters provided by the NWP system may be difficult to quantify and model Conflict between having enough cloud variables for an accurate RT calculation while limiting the number of cloud variables to those that can be uniquely estimated in the analysis from the observations

Clear population Cold departures indicating cloud contamination in OBS Observed radiance at 11 um minus radiance calculated from background in clear sky

Potential difficulties The cloud uncertainty may be an order of magnitude larger than the T and Q signal (i.e. 10s of kelvin compared to 0.1s of kelvin) The radiance response to cloud changes is highly non- linear (i.e. H(x) = H x (x)) Errors in background cloud parameters provided by the NWP system may be difficult to quantify and model Conflict between having enough cloud variables for an accurate RT calculation while limiting the number of cloud variables to those that can be uniquely estimated in the analysis from the observations

surface full cloud at 500hPa dR/dT 500 = 0 dR/dT* = 1 dR/dT 500 = 1 dR/dT* = 0 Clear and Cloudy Jacobians (impact at the cloud top)

Potential difficulties The cloud uncertainty may be an order of magnitude larger than the T and Q signal (i.e. 10s of kelvin compared to 0.1s of kelvin) The radiance response to cloud changes is highly non- linear (i.e. H(x) = H x (x)) Errors in background cloud parameters provided by the NWP system may be difficult to quantify and model Conflict between having enough cloud variables for an accurate RT calculation while limiting the number of cloud variables to those that can be uniquely estimated in the analysis from the observations

Observed radiance at 11 microns minus radiance calculated from NWP cloud background profile Many clouds with significant radiance signals are accurately represented by the NWP model and RT modelled !

Potential difficulties The cloud uncertainty may be an order of magnitude larger than the T and Q signal (i.e. 10s of kelvin compared to 0.1s of kelvin) The radiance response to cloud changes is highly non- linear (i.e. H(x) = H x (x)) Errors in background cloud parameters provided by the NWP system may be difficult to quantify and model Conflict between having enough cloud variables for an accurate RT calculation while limiting the number of cloud variables to those that can be uniquely estimated in the analysis from the observations

Choice of cloud parameters and ambiguity with T and Q P n T T P n A very simple cloud model (e.g. single layer grey cloud amount and pressure) should more readily estimated from the data (independently of T and Q), but will make the forward RT calculation very inaccurate in many cloud conditions A more complex cloud model (e.g. cloud liquid and ice at each model level) will allow a more forward RT calculation, but may be difficult to estimate independently of T and Q and may alias into erroneous increments

Two approaches to assimilate cloud affected infrared radiances

Simplified system: very simple cloud representation currently limited to overcast scenes no information on clouds taken from model no back interaction with model via physics Advanced system: very complex cloud representation all cloud conditions treated information on clouds taken from model back interaction with model via physics X=(T,Q,V,ciw,clw,cc) X=(T,Q,V,cp,cf) cp cf

Experience with simplified Cloudy IR Radiance Assimilation ( only overcast conditions used ) X=(T,Q,V,cp,cf) cp cf See McNally (2009) QJRMS 135, for more details…

Why overcast scenes…?

The cost function J (X) model state has only one extra variable = cloud top height

Why use cloudy radiances only in overcast conditions ? Overcast clouds are least ambiguous in the radiance data Cloud control vector collapses to a single number (cp) Problems with cloud overlap assumptions vanish No cross-talk between cloud and surface variables Termination of jacobians at cloud top provides new high vertical resolution Information on temperature

Impact of overcast data on the ECMWF analysis…

surface full cloud at 500hPa dR/dT 500 = 0 dR/dT* = 1 dR/dT 500 = 1 dR/dT* = 0 Clear and Cloudy Jacobians (impact at the cloud top)

Temperature increments above low clouds Overcast data coverage from HIRS, AIRS and IASI and the estimated cloud top pressure

Temperature increments above low clouds Analysis temperature increments at 700hPa Overcast data coverage from HIRS, AIRS and IASI and the estimated cloud top pressure

Temperature increments above high clouds Overcast data coverage from HIRS, AIRS and IASI and the estimated cloud top pressure

Temperature increments above high clouds Analysis temperature increments at 250hPa Overcast data coverage from HIRS, AIRS and IASI and the estimated cloud top pressure

…do these extra increments at the cloud top do any good..?

Improved analysis fit to isolated radiosonde observations Monthly averaged RMS temperature increment difference (CLOUDY minus CTRL). Shaded areas indicate a reduction in increments in excess of 0.1K when the cloud radiances are assimilated.

…forecast impact...?

Cloud obscured singular vector ? In this case the use of overcast observations resulted in analysis differences in an area suggested to be sensitive by the singular vector locations 500hPa temperature analysis difference (K) ? CNTRL CLOUDY Location of leading 500hPa singular vectors SH 500hPa Z Extra overcast data used compared to CTRL

Towards an Advanced Cloudy IR Radiance Assimilation

(T,Q,V)(T,Q,V) model physics (M) (T,Q,V,ciw,clw,cc) Cloudy (RT) (R cal ) Jo=(R obs -R cal ) Cloudy (RT) * model physics (M*) (T,Q,V,ciw,clw,cc)* (T,Q,V)* (R cal )* Cloudy radiances R cal are simulated via a chain of forward operators (M,RT). The fit of the analysis to the observations is computed (Jo) Jo is minimized by perturbing the analysis variables according to gradients from a chain of adjoint operators (RT*,M*)

…not yet… (at least for the infrared)

Adjusting analysis variables to fit the cloudy radiance observations

End