CLEAN-AP Update Cl utter E nvironment An alysis using A daptive P rocessing Sebastián Torres and David Warde CIMMS/The University of Oklahoma and National Severe Storms Laboratory/NOAA NEXRAD TAC Norman, OK 29 August, 2012

CLEAN-AP is a novel real-time, automatic, integrated technique for ground clutter detection and filtering that produces meteorological data with improved quality while meeting NEXRAD technical requirements. what is CLEAN-AP? 2

While CLEAN-AP can deal with normal- (NP) and anomalous-propagation (AP) clutter, it is not a mitigation technique for moving clutter such as airplanes, cars, or wind turbines. what is CLEAN-AP not? 3

Developed concept (Spring ‘08) –Filed invention disclosure with OU’s OTD Evaluated performance using simulations (Spring ‘08) Implemented in real time on the NWRT PAR (Fall ‘08) Compared with current WSR-88D clutter mitigation (Fall ‘09) –KEMX (Tucson, AZ), KTLX (Oklahoma City, OK), KABX (Albuquerque, NM), KCRI (ROC testbed) Presented at November 2009 TAC meeting –initially presented CLEAN-AP “… the technique shows potential utility…”, “… encouraged by early results…” “… [have not] shown the scientific details behind the algorithm…” “… [need] case comparisons with CMD…” Extended to dual-polarization (Spring ‘10) –KOUN (S-band) and OU’ (C-band) Developed clutter model (Spring ‘10) –Form/Function/Fit for RVP8 milestones (I) 4

Presented at fall 2010 ROC/NSSL/NCAR TIM –presented more technical details and data cases –received positive feedback from ROC eng and agency reps Delivered CLEAN-AP algorithm description (Fall ’10) –NSSL Report 14 (confidential attachment) Extended to staggered PRT (Fall ‘10) Presented at March 2011 TAC meeting –decision briefing “The TAC members agreed to move forward with conducting an engineering evaluation on CLEAN AP.” Documented CLEAN-AP performance (Winter ’11) –NSSL Report 15 Delivered SPRT CLEAN-AP algorithm description (Spring ’12) –NSSL Report 16 (confidential) milestones (II) 5

Ground Clutter Filtering Ground Clutter Detection Unfiltered data Filtered data clutter mitigation at the RDA Control CMD GMAP CLEAN-AP 6

CLEAN-AP’s core: the ASD lag-1 autocorrelation spectral density For periodic signals: F 1 (k) = F 0 (k) e j2  k/M –S 1 (k) = F 0 *(k) F 0 (k) e j2  k/M =S (k) e j2  k/M –the ASD is the PSD with linear phase sample time 10 M2M2M1M1 V0V0 V1V1 window & DFT F1F1 F0F0 S 1 (k) = F 0 * (k)F 1 (k)

MeasuredTrue v v ASD v v v PSD Ground Clutter v PSD Weather v v ASD v v Leakage nothing is ideal… Spectral leakage in the measured ASD is the basis for clutter detection in CLEAN-AP

Filtering Detection how does CLEAN-AP work? Integrated Detection and Filtering Estimate CNR Select appropriate data window Compute ASD Identify components with clutter contamination –Phase of ASD is near zero due to leakage effect Remove clutter Reconstruct weather ASD is used to estimate meteorological variables v v |ASD| zero phase DC Arg(ASD) PSD 9

Performance Summary

data window selection CLEAN-AP vs. GMAP More aggressive data windows contain spectral leakage but increase variance of estimates better suppression lower variance GMAP % 11

notch width selection CLEAN-AP vs. GMAP CLEAN-AP uses magnitude and phase for improved notch width determination better suppression lower bias NW determination magphase GMAP CLEAN-AP 12

13 CLEAN-AP has more clutter suppression and less variance of estimates Clutter Suppression Doppler Mode CLEAN-APGMAP

14 CLEAN-AP has a more robust notch width determination that results in smaller biases Reflectivity (Power) Bias Doppler Mode CLEAN-AP GMAP

Case Examples

Ground-clutter suppression –Due to the mountainous terrain close to the radar, KEMX data exhibited “hot spots” (clutter residue) after the initial release of CMD (B11). This led to a few software changes (B11 fixed) KEMX Tucson, AZ Can you see the mountains? 16

KEMX Reflectivity Unfiltered CMD+GMAP (B11 fixed) CLEAN-AP CLEAN-AP mitigates both weak and strong clutter contamination 17

KEMX Tucson, AZ CLEAN-AP Performance CLEAN-AP adapts to clutter environment 18 CLEAN-AP Power Removed CLEAN-AP Window Used

Zero-isodop loss –Weather with narrow spectrum width and near zero velocity has nearly the same spectrum as clutter KCRI Norman, OK What happened to the zero-isodop? if it looks like a duck… 19

20 KCRI Reflectivity Unfiltered CLEAN-AP CMD+GMAP (B11 fixed) CLEAN-AP better preserves the zero isodop

21 KCRI Velocity CLEAN-AP better preserves the zero isodop Unfiltered CLEAN-AP CMD+GMAP (B11)

Support implementation on the ORDA Support engineering evaluation Investigate using dual-pol information to improve detection performance Investigate using H and V channels to determine the filter’s notch width –This also applies to the current filter Future Work

CLEAN-AP is a novel real-time, automatic, integrated technique for ground clutter mitigation that produces meteorological data with improved quality while meeting NEXRAD technical requirements. In March 2011, the TAC endorsed its engineering evaluation on the WSR-88D. Summary 23

back up slides 24

CLEAN-AP is automatic –no need for user intervention –real-time ground-clutter detection –no need for clutter maps CLEAN-AP produces data with improved quality –adaptive data windowing finds a good compromise between clutter suppression and data quality CLEAN-AP meets NEXRAD requirements –improved clutter suppression –requirements (Z) met with as few as 8 samples CLEAN-AP is integrated –one algorithm for ground clutter detection and filtering –gate-by-gate operation the CLEAN-AP filter (I) 25

CLEAN-AP “sets the stage” for further spectral processing –phase provides useful information –immune to biases from circular convolution CLEAN-AP is operational on the NWRT PAR –running in real-time since Sep 2008 –performance informally evaluated by meteorologists and forecasters (PARISE experiments) CLEAN-AP is compatible with current and future upgrades –SZ-2, super resolution, dual pol., hybrid SW, –SPRT, range oversampling, etc. the CLEAN-AP filter (II) 26

KEMX Clutter Map (Pwr Rmvd > 10 dB) Clutter Map (Pwr Rmvd > 20 dB) 27

Snow event with embedded storms KTLX Oklahoma City, OK Where is the zero? 28

29 KTLX Reflectivity UnfilteredCLEAN-AP Where is the zero isodop?

30 KTLX Velocity UnfilteredCLEAN-AP No detectable zero isodop loss

KOUN What about dual polarization? Reflectivity Differential Reflectivity Correlation Coefficient Differential Phase CLEAN-AP is compatible with dual polarization 31