1. 2. Description of MIIDAPS 1. Introduction A Generalized Approach to Microwave Satellite Data Assimilation Quality Control and Preprocessing Here, we present recent efforts supported by the JCSDA to advance and increase satellite observations assimilated within the GSI analysis system used to initialize both the Global Forecast System (GFS) model and regional Hurricane WRF (HWRF) model at NOAA NCEP. Specifically, the use of the Multi-Instrument Inversion and Data Assimilation Preprocessing System (MIIDAPS) within the GSI will be discussed. This 1DVAR preprocessor is applicable to current and future microwave satellite sounders and imagers including those from POES and MetOp AMSU-A and MHS, NPP/JPSS ATMS, DMSP F16-F20 SSMI/S, and was recently extended to GCOM-W1 AMSR2. The capability of the 1DVAR preprocessor includes increased quality control of the microwave radiances to be assimilated, and also to provide surface emissivity or hydrometeor (cloud, rain) information to the assimilation system which may increase the number and types of observations that can be assimilated. The information provided by the 1DVAR preprocessor could be considered complimentary to ongoing GSI development efforts associated with the assimilation of surface sensitive channels over non-ocean surfaces as well as cloudy radiance assimilation. Advancement in the assimilation of these types of observations should have significant positive impact on both global NWP forecast and regional NWP and tropical cyclone track and intensity forecasts. Kevin Garrett 1 and Sid Boukabara 2 1. Riverside Technology, Inc., JCSDA 2. NOAA/NESDIS/STAR, JCSDA 3. Preliminary Assessments 4. Other Potential Utility5. Conclusions Run MIIDAPS Measured Radiances CRTM Initial State Vector Simulated Radiances Comparison: Fit Within Noise? No Update State Vector New State Vector Solution Reached Yes GSI / setuprad module Apply Quality Control to observations – cloud detection, surface sensitivity, bias correction, gross obs checks, other QC Read in guess fields and observations The 1DVAR algorithm applied here is also known as the Microwave Integrated Retrieval System (MiRS). It is based on a physical approach using the CRTM for its forward and jacobian operators. As such, the retrieval is valid in all cases the CRTM is valid in, including all weather conditions and over all surface types. The state vector supports the retrieval of temperature, water vapor, and hydrometeor profiles, along with surface emissivity and skin temperature depending on the information content of the sensor data being applied. A post-processor then derives products from the core-retrieval, such as TPW or Rainfall Rate. For integration with the GSI, the 1DVAR algorithm has been converted to a subroutine and interfaced with the setuprad module. Figure 3. MIIDAPS convergence metric (ChiSq) (left) and number of iterations used during the retrieval (right). Both metrics provide information on difficulty to assimilate observations. ChiSq values < 1 denote convergence of the variational algorithm. To reach the iterative solution, the algorithm seeks to minimize the cost function where X in the 1 st term on the right is the retrieved state vector, and the term itself represents the penalty for departing from the background X 0, weighted by the error covariance matrix B. The 2 nd term represents the penalty for the simulated radiances Y departing from the observed radiances Y m, weighted by instrument and modeling errors E. This leads to the iterative solution, where ∆X is the updated state vector at iteration n+1, and K is the matrix of Jacobians which contain the sensitivity of X (parameters to retrieve) to the radiances. Figure 2. MIIDAPS core and post-processing products. Core products are retrieved simultaneously as part of the state vector. VIPP products are derived through vertical integration (hydrometeors), catalogs (SIC, SWE), or fast regressions (Rain Rate). Figure 3. The MIIDAPS outputs several metrics useful for data assimilation, including quality control flags, convergence metrics, and averaging kernel. Core Products with Vertical Integration and Post-Processing (VIPP) 1DVAR Outputs Vertical Integration Post Processing (Algorithms) TPW RWP IWP CLW -Sea Ice Concentration -Snow Water Equivalent -Snow Pack Properties -Land Moisture/Wetness -Rain Rate -Snow Fall Rate -Wind Speed/Vector -Cloud Top -Cloud Thickness -Cloud Phase Core Products (EDR) Temp. Profile Humidity Profile Emissivity Spectrum Skin Temperature Cloud Amount Prof Ice Amount Prof Rain Amount Prof VIPP Quality Control Outputs QC Summary (Good, Bad, Rain, Cloud) Cloud Amount Surface Emissivity Chi-squared (< 1 Convergence reached) Averaging Kernel Contribution Function Uncertainty Matrix Useful for Radiance Assimilation Useful for Product Assimilation 1DVAR Emiss CRTM Emiss 31 GHz EmissivityObs. - BackgroundGSI Quality Flags A 1DVAR preprocessor can provide information on cloud liquid water content (CLW) to flag cloudy/rainy radiances for QC. For clear-sky radiance assimilation, it is necessary and critical to remove all observations containing clouds and precipitation since the radiative transfer model (RTM) biases are not well characterized in these conditions (nor are they equivalent to clear-sky radiance bias). Figure 7. a) 1DVAR retrieved CLW from NOAA-18, b) collocated GFS 6-hr forecast CLW, c) difference between 1DVAR and GFS cloud. Histograms of the dTB for AMSU-A channel 1 (d) and MHS channel 2 (e) using the GFS for cloud filters (black) and 1DVAR for cloud filters (red). Using enhanced cloud filtering of the 1DVAR shows generally narrower histograms with less bias, particularly at 157 Ghz. Operationally, the GSI uses an alternate brightness temperature-based cloud detection algorithm. e) d) a) b) c) A 1DVAR preprocessor can provide information on cloud, rain and ice to flag cloudy/rainy radiances for cloudy/rainy radiance or rainfall rate assimilation. Passive microwave radiances are sensitive to non- precipitating cloud over ocean, and most precipitation events over ocean and land. 1DVAR retrieved hydrometeor amounts can be used to derive instantaneous rainfall rates to be used as observations in data assimilation. Assimilated rainfall rates can then be used to reverse parameterize model state variables and reduce forecast spin-up times as well as improve accuracy. Figure 8. Top) 1DVAR retrieved cloud and rain water path from GCOM-W1 AMSR2 over Hurricane Sandy on Oct. 27, 2012. Bottom) Vertical cross section of 1DVAR retrieved rain and ice profile along longitude -75 °. The freezing line from the retrieved temperature profile is plotting in pink, along with the top rain and bottom ice layers in blue and brown, respectively. Figure 1. Schematic of the MIIDAPS retrieval algorithm iterative process. The initial state vector can either come from a climatology or the NWP analysis background field (6 hr forecast) itself. The MIIDAPS has been implemented in the GSI for the Suomi NPP ATMS sensor. It is called for each “setuprad” instance on the thinned set of ATMS radiances before any quality control. It is highly tunable to control the types of products and QC information to return to GSI, as well as for resource (CPU) considerations (restricting number of attempts/iterations/channels used). Figure 4. Residual of MIIDAPS retrieved state vector (Observation minus Simulation) for ATMS channel 2 (left) and the GSI analysis Observation minus Analysis (O-A). The MIIDAPS residual shows nearly global fit to the observations. The GSI O-A filed shows higher residuals especially over non-ocean surfaces and in cases of precipitation. Figure 5. Histogram of the residuals shown in Figure 4 for ATMS channel 2 for all points which pass QC MIIDAPS ChiSqMIIDAPS No. of Iterations Figure 6. Row a) MIIDAPS retrieved 31 GHz emissivity, GSI O-B using MIIDAPS emissivity in the background field, and GSI QC flags (left to right), Row b) Same as Row a but using CRTM/physical emissivity, Row c) difference between MIIDAPS and CRTM/physical emissivity and difference between O-B, Row d) Histograms of the O-B using the 1DVAR emissivity (left) and CRTM/physical emissivity (right). Large differences are noted in surface emissivity between the MIIDAPS and physical model, particularly over N. Africa and sea-ice, which in turn relate to large differences in the O-B which influences the gross error check in the GSI (QC Flag 3). The histograms show a more normal distribution of O-B (before bias correction) when using MIIDAPS emissivity in the GSI forward simulation of the background field compared with the physical model, slightly more points that pass QC, and similar bias/error. A 1DVAR preprocessor has been implemented in the GSI to apply to passive microwave observations (preliminarily SNPP ATMS). MIIDAPS 1DVAR fits observations with narrow residual error distribution compared to GSI which must fit multiple observation types and is balanced by tighter background constraints. Use of the retrieved emissivity in place of physical model emissivity has shown increase in observations assimilated. Cycling is required to adjust bias correction coefficients before assessing impact on O-A and forecasts. MIIDAPS retrieved cloud field shows large displacement relative to the background GFS forecast fields. 6. Future Work Assess using CLW and MIIDAPS QC metrics in the analysis to determine which observations to assimilate and their weights. Transfer background temperature and humidity profiles to MIIDAPS vertical pressure grid to use as first-guess in retrieval. Turn on MIIDAPS for other sensors, including for upcoming SSMI/S and AMSR2 imaging channel assimilation. Extend to GPM GMI when test data become available. Extend the MIIDAPS to hyperspectral IR sensors. Contact Information: Kevin.Garrett@noaa.gov