Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Chemcial Data Assimilation for Air Quality Forecasting Methods used in Europe H. Elbern with support from 1 A. Strunk, L. Niradzik, E. Friese, Z. Milbers, 2 M. Schrödter-Homscheidt 1 Rhenish Institute for Environmental Research at the University of Cologne (RIU) and 1 Helmholtz virt. Inst. for Inverse Modelling of Atmospheric Chemical Composition (IMACCO) 2 German Data Centre for Remote sensing (DLR-DFD)

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Contents: 1.Survey and scope 2.Air Quality data assimilation in routine operations today 3.Advanced scientific applications –Kalman filter –4dimensional variational data assimilation (4Dvar) –Example applications 4.Future research directions –Short term (1-3 years) –Long term (> 3 years)

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Survey (1) Regional operational data assimilation to comply with EU legislation core assimilation data of in situ type from observation networks operated by national or regional/state authorities Air Quality considered as a high resolution problem  limited area and nested models 2 major European efforts in Global Environmental Monitoring and Security (GEMS = European GEOSS) –ESA funded PROMOTE ( –EU funded GEMS (new!) 1 Survey an Scope

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 European metropolitan chemograms ozone NO2 PM10 SO2 CO benzene 1 Survey an Scope

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Prevair system in PROMOTE Method:Kriging Reference: Blond et Vautard, JGR, 2003, 2004 analysis and observations forecast and observations 2 Routine operations

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 EURAD system in PROMOTE Method: 2 and 3Dvar Reference:Reduced version of Elbern and Schmidt, JGR, 2001 analysis and observations forecast and observations 2 Routine operations

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Survey (2) Global tropospheric partly operational MOCAGE (Meteo France) 3DFGAT, (= 3Dvar with “First Guess at Appropriate Time”) TM3-5 (KNMI) “Kalman filter” with parameterize covariance update, TM3-BETHY (MPI-BGC) soil column variational (1 spatial D + time D) 2 Routine operations

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Special challenges of tropospheric chemistry data assimilation The following problems prevail: 1.strong influence of manifold processes including emissions and deposition 2.spatially highly variable “chemical regimes” 3.chemical state observability (= “analyseability”) hampered by manifold hydrocarbon species 4.consistency with heterogeneous data sources: satellite data and in situ observations 3. Advanced Mthods General remarks This invokes the application space-time data assimilation algorithms BLUE preserving the BLUE property (Best Linear Unbiased Estimator)

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Advanced spatio-temporal data assimilation: 4D dimensional variational and Kalman filtering Model not to passively accept external 3D analyses (probably engendering spurious relaxations) Rather contribute with its dynamics/ chemical kinetics as constraint to estimate the most probable state or parameter values  Best Linear Unbiased Estimate (BLUE) while using data and models consistently allows for Hypothesis testing 3. Advanced Mthods General remarks

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Advanced spatio-temporal methods used in tropospheric chemistry data assimilation 4-dim variational data assimilation (4D-VAR, and versions: 4D physical space statistical analysis system (Amodei, 1995; Courtier, 1997); 4D-var incremental (Courtier et al, 1994)) reduced complexity versions of the Kalman Filter: Reduced Rank Square root KF (RRSQKF, Verlaan and Heemink, 1997), ensemble KF (Evensen, 1994), SEEK, SEIK, (Verron et al., 1999) Spacio-temporal BLUEs applied in tropospheric chemistry data assimilation: 4D var: –with EURAD (Elbern and Schmidt, 1999, 2001), –with POLAIR (Issartel and Baverel, 2003) RRSQKF –with LOTOS model (van Loon et al, 2000), –with EUROS model (Hanea et al. 2004) Remark: 3D BLUE algorithm analyses like those from Optimal Interpolation, once ingested into a model, do not result in a 4D BLUE analysis For example 3D-FGAT is not! 3. Advanced Mthods General remarks

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Hypothesis: initial state and emission rates are least known emission biased model state only emission rate opt. only initial value opt. true state observations time concentration joint opt. 3. Advanced Mthods General remarks

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Kalman filter, basic equations 3. Advanced Mthods Lakman filter

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Reduced rank Kalman filter (basic idea) Appoximate covariance matrices P b,a (n x n) by a product of suitably low ranked matrix S b,a (n x p), and p << n. Same procedure for the system noise matrix Q with T (n x r). The forecast step remains unchanged. The forecast error covariance matrix rests on 2 x p model integrations only! The enlargement of p by r enforces periodic reductions of columns (with lowest ranked eigenvalues) 3. Advanced Mthods Lakman filter

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods Lakman filter Reduced rank Kalman filter (calculus) In practice, all calculations can be performed without actually calculating matrices P! Positive semi- definitenes is maintained!

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Air quality Kalman filter implementations and comlexity reduction strategy applied LOTOS model (van Loon et al., 2000) –reduced rank square root EUROS model (Hanea et al., 2004) –ensemble –reduce rank square root 3. Advanced Mthods Lakman filter Methods used with optimisation of various parameters including emissions

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Transport-diffusion-reaction equation and its adjoint 3. Advanced Mthods 4D var

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods 4D var 3. Advanced Mthods Lakman filter 3. Advanced Mthods Lakman filter In the troposphere for emission rates the product (paucity of knowledge*importance) is high

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Adjoint integration “backward in time” (slide from lecture 1) How to make the parameters of resolvents i M(t i-1,t i ) available in reverse order??

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 The 4D variational method: Key development: construction of the adjoint code backward model (backward differential equation) adjoint algorithm (adjoint solver) adjoint code 3. Advanced Mthods 4D var forward model (forward differential equation) algorithm (solver) code

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Source identification by 4Dvar POLAIR model Issartel and Baverel Ecole National de Ponts et Chausees (ENPC) 3. Advanced Mthods 4D var application Calculations of optimal retroplumes (Issartel and Baverel, ACP, 2003)

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods 4D var application 4D-var configuration Mesoscale EURAD 4D-var data assimilation system meteorological driver MM5 meteorological driver MM5 EURAD emission model EEM emission 1. guess EURAD emission model EEM emission 1. guess direct CTM emission rates Initial values Initial values minimisation adjoint CTM adjoint CTM observations analysis gradient fore- cast

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Computational complexity estimate of the variational algorithm Storage strategy # forward runs/iteration storage Complexity [T forw ] total storage 1 const*N x *N y *N z *N c *N T *N o O( ) 3 operatorwise 1 level 2 N x *N y *N z *N c *N T * N o O( ) 4 dynamic stepwise 2 levels 3 N x *N y *N z *N c *N T O( ) 5 Nx*Ny*Nz spatial dimensions O( ) N c # constituents O(100) N T # time steps of assimilation window O(10-100) N o # operators O(10) constintermediate results O(10 4 ) 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 backward integration 1. step2. step3. step4. step5. step6. step7. step8. step9. step 2 level forward and backward integration scheme time direction disk forward integration backward integration intermediate storage Level 1 intermediate storage Level 1 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 horiz. Advekt x horiz. Advekt y vert. Diffusion vert.. Advekt Adjungierte horiz. Advekt x Adjungierte horiz. Advekt y Adjungierte vert. Advekt Adjungierte vert.Diffusion horiz. Advekt x horiz. Advekt y vert. Diffusion vert.. Advekt Adjungierte horiz. Advekt x Adjungierte horiz. Advekt y Adjungierte vert. Advekt Adjungierte vert.Diffusion Chemie Adjungierte Chemie storage sequence: level 2, operator split (each time step) intermediate storage Level 2 horiz. Advekt x horiz. Advekt y Chemie vert. Diffusion vert.. Advekt horiz. Advekt x horiz. Advekt y vert. Diffusion vert.. Advekt horiz. Advekt x horiz. Advekt y vert. Diffusion vert.. Advekt Adjungierte horiz. Advekt x Adjungierte horiz. Advekt y Adjungierte vert. Advekt Adjungierte vert.Diffusion horiz. Advekt x horiz. Advekt y vert. Diffusion vert.. Advekt Adjungierte horiz. Advekt x Adjungierte horiz. Advekt y Adjungierte vert. Advekt Adjungierte vert.Diffusion Chemie Adjungierte Chemie horiz. Advekt x horiz. Advekt y Chemie vert. Diffusion vert.. Advekt horiz. Advekt x horiz. Advekt y vert. Diffusion vert.. Advekt intermediate storage Level 1 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Computational resources requested with a 14 h assimilation interval about 18 iterations requested results in 12 CPU-hours with 121 processors of a T3E 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Treatment of the inverse problem to infer emission rates

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Normalised diurnal cycle of anthropogenic surface emissions f(t) emission(t)=f(t;location,species,day) * v(location,species) day in {working day, Saturday, Sunday}v optimization parameter 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Semi-rural measurement site Eggegebirge assimilation intervalforecast 7. August 8. August observations no optimisation initial value opt. emis. rate opt. joint emis + ini val opt. 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 error statistics bias (top), root mean square (bottom) assimilation window forecast assimilation window 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005  x=54 km Which is the requested resolution? BERLIOZ Which is the requested resolution? BERLIOZ grid designs and observational sites (20.  )  x=18 km  x=6 km  x=2 km Control and diagnostics

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods 4D var application Some BERLIOZ examples of NOx assimilation Time series for selected NOx stations (upper panel NO, lower panel NO 2 ) on nest 2. + observations, -- - no assimilation,____ N1 assimilation (18 km), ____N2 assimilation.(6 km), -grey shading: assimilated observations, others forecasted. NO NO 2

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods 4D var application Emission source estimates by inverse modelling Optimised emission factors for Nest 3 NO 2, (xylene (bottom), CO (top), and SO 2 (from left to right). surface height layer ~32-~70m

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Status: EURAD assimilation system prepared for 4D-var assimilation production run 1.7.  Gas phase satellite observations –KNMI - NO2 tropospheric columns, SCIAMACHY averaging kernel assimilation –NNORSY neural network based ozone tropospheric columns (revised version based on GOME) –IMK - MIPAS upper troposphere Aerosol phase satellite observations –DFD - PM10 bulk retrievals SCAMACHY-AATSR 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods 4D var application NO2 tropospheric column assimilation IFE: Example: BERLIOZ episode GOME retieval Courtesy: A. Richter, IFE Bremen no assimilation with assimilation NOAA 14 11:51 UTC

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods 4D var application IUP Bremen GOME  GOME forecast validation model/retrieval ratio for BERLIOZ 20. (assimilated) + 21.(forecasted) ideal no assimilationwith assimilation verification forecast (not assimilated) forecast

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 KNMI GOME  GOME forecast validation model/retrieval ratio for BERLIOZ 20. (assimilated) + 21.(forecasted) no assimilationwith assimilation ideal verification forecast (not assimilated) forecast 3. Advanced Mthods 4D var application

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods 4D var application GOME ozone profile assimilation BERLIOZ ( ) Data: Neuronal Network retrieval (Müller et al., 2003) + NNORSY ozone retrieval first guess after assimilation

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Advanced Mthods 4D var application Ozone observation increment layer: hPa radius of influence: 200 km height dependent influence radii

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Future Research directions: short term operationalisation of advanced methods: use of complexity constraining algorithms aerosol data assimilation Model Output Statistics (MOS) 4. Future Methods short term

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Future Methods short term (NH 4 )2SO 4 watersoluble NH 4 NO 3 watersoluble H 2 SO 4 watersoluble HNO 3 watersoluble H 2 Owatersoluble prim. organicinsoluble Prod. Aromatewatersoluble Prod. Alkanewatersoluble Prod. Alkenewatersoluble Prod.  -pinenewatersoluble Prod. limonenewatersoluble elem. carbonSOOT prim. small aerosolsinsoluble maritimeSEASALT mineralMINERAL TRANSPORTED other anthropogenicinsoluble MADESYNAER WASO INSO The attribution EURAD-MADE-SORGAM model variables to SYNAER aerosol types 5 Aerosol types can be distinguished Courtesy M.Schrödter-Homscheidt, DFD joint SCIAMACHY-AATSR retrieval (Holzer-Popp et al. JGR, 2003)

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, Future Methods short term MADE, Level 1, SO4, accumulation mode, mass.-conc.  g/m³ example 5. August 1997 Each MADE-variable modified according to the minimisation result, relative structure of the aerosol vertical profile remains unaffected background mass conzentration analysis increment new mass conzentration Colaboration M.Schrödter-Homscheidt, DFD

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Future Research directions: long term Multi scale assimilation (for error of representativity reduction)  dynamical nesting Systems with non-Gaußian error characteristics Systems with phase transitions (microphysics) advanced a posteriori-evaluations

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 necessary condition with p observations: minimum should be half the number of observations: Necessary a posteriori condition for general consistency test (  2 test) With 3. Advanced Mthods General remarks

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 Objective function as a sum of different data sources Partition to any data source possible? (Ivanov and Deque, 2001) m j number of observations from information source/satellite j S j source j error covariance matrix P a analysis error covariance matrix

Workshop on Cheemical Data Assimilation and Data Needs for Air Quality Forecasting Silver Springs, MD, June 20-22, 2005 The End The End Thank you for attention