Analysis of Satellite-Based TC Intensity Estimation in the WNP

Slides:

Advertisements

Similar presentations

Operational Use of the Dvorak Technique at the NHC

Advertisements

Recent Advances in the Processing, Targeting and Data Assimilation Applications of Satellite-Derived Atmospheric Motion Vectors (AMVs) Howard Berger 1,

1 GOES-R Hurricane Intensity Estimation (HIE) Validation Tool Development Winds Application Team Tim Olander (CIMSS) Jaime Daniels (STAR)

Yasi vs. Larry: What the Satellite Saw John Knaff NOAA/NESDIS –RAMMB Fort Collins, Colorado.

Joint Typhoon Warning Center Forward, Ready, Responsive Decision Superiority UNCLASSIFIED An Overview of Joint Typhoon Warning Center Tropical Cyclone.

Future Plans  Refine Machine Learning:  Investigate optimal pressure level to use as input  Investigate use of neural network  Add additional input.

Future Plans  Refine Machine Learning:  Investigate optimal pressure level to use as input  Investigate use of neural network  Add additional input.

Hurricane center-fixing with the Automated Rotational Center Hurricane Eye Retrieval (ARCHER) method Tony Wimmers, Chris Velden University of Wisconsin.

Microwave Imagery and Tropical Cyclones Satellite remote sensing important resource for monitoring TCs, especially in data sparse regions Passive microwave.

CORP Symposium Fort Collins, CO August 16, 2006 Session 3: NPOESS AND GOES-R Applications Tropical Cyclone Applications Ray Zehr, NESDIS / RAMM.

Exercise – Constructing a best track from multiple data sources NATIONAL HURRICANE CENTER JACK BEVEN WHERE AMERICA’S CLIMATE AND WEATHER SERVICES BEGIN.

Forecasting Tropical cyclones Regional Training Workshop on Severe Weather Forecasting and Warning Services (Macao, China, 9 April 2013)

Kenji KISHIMOTO Forecast Division Japan Meteorological Agency.

Impact of the 4D-Var Assimilation of Airborne Doppler Radar Data on Numerical Simulations of the Genesis of Typhoon Nuri (2008) Zhan Li and Zhaoxia Pu.

ATMS 373C.C. Hennon, UNC Asheville Observing the Tropics.

An airborne portable expendable probe receiver/processor system for operational acquisition and transmission of ocean observations from WC-130J tropical.

Comparison of Airborne SFMR, Best Track and Dvorak ADT Maximum Surface Wind Estimates in Atlantic Tropical Cyclones Peter G. Black (1), Stephanie Mullins.

TC Intensity Estimation: SATellite CONsensus (SATCON) Derrick Herndon, Chris Velden, Tony Wimmers, Tim Olander International Workshop on Tropical Cyclone.

JTWC SATOPS Challenges Capt Kathryn Payne 28 April 2009.

April nd IBTrACS Workshop 1 Operational Procedures How can we build consistent, homogeneous, well- documented climate quality data?

CIMSS TC Intensity Satellite Consensus (SATCON) Derrick Herndon and Chris Velden Meteorological Satellite (METSAT) Conference Ford Island Conference Center.

STATISTICAL ANALYSIS OF ORGANIZED CLOUD CLUSTERS ON WESTERN NORTH PACIFIC AND THEIR WARM CORE STRUCTURE KOTARO BESSHO* 1 Tetsuo Nakazawa 1 Shuji Nishimura.

JTWC Satellite Operations (SATOPS) Capt Kathryn Payne 27 April W SINLAKU ATCF.

30 November December International Workshop on Advancement of Typhoon Track Forecast Technique 11 Observing system experiments using the operational.

A Comparison of Two Microwave Retrieval Schemes in the Vicinity of Tropical Storms Jack Dostalek Cooperative Institute for Research in the Atmosphere,

TCS-08 Near Real-Time Satellite Field Program Support and Validation [Jeff Hawkins, Kim Richardson, Tom Lee, Buck Sampson, Steve Miller, Arunas Kuciauskas]

The ARCHER automated TC center-fixing algorithm: Updates on real-time operations, accuracy and capabilities Anthony Wimmers and Christopher Velden Cooperative.

NHC/JHT Products in ATCF Buck Sampson (NRL, Monterey) and Ann Schrader (SAIC, Monterey) IHC 2007 Other Contributors: Chris Sisko, James Franklin, James.

DATA ASSIMILATION FOR HURRICANE PREDICTION Experimental system and results of semi-operational implementation during the 2010 Atlantic Hurricane Season.

A. FY12-13 GIMPAP Project Proposal Title Page Title: Improvements to the Advanced Dvorak Technique Status: New – but continuing work from GIMPAP FY07-09.

I n t e g r i t y - S e r v i c e - E x c e l l e n c e Headquarters U.S. Air Force 1 Air Force Director of Weather 1 March 2010 USAF Tropical Cyclone.

AMSU Product Research Cooperative Institute for Research in the Atmosphere Research Benefits to NOAA: __________________ __________________________________________.

The Potential for Improved Short-term Atlantic Hurricane Intensity Forecasts Using Recon-based Core Measurements Andrew Murray, Robert Hart,

CIMSS/NESDIS-USAF/NRL Experimental AMSU TC Intensity Estimation: Storm position corresponds to AMSU-A FOV 8 [1 30] Raw Ch8 (~150 hPa) Tb Anomaly: 5.36.

F l y - F i g h t - W i n HQ United States Air Forces Director AF Weather 3 Mar 08 USAF Tropical Cyclone Support.

AMS Annual Meeting - January NRL Global Model Adaptive Observing During TPARC/TCS-08 Carolyn Reynolds Naval Research Laboratory, Monterey, CA OUTLINE:

A Proposed New Strategy for Tropical Cyclone Reconnaissance Based on Western Pacific TCS08 Proof of Concept Peter G. Black (1), Jeffrey D. Hawkins (2)

AODT The Advanced Objective Dvorak Technique JHT Progress Report - Latest Advancements Timothy Olander, Christopher Velden, James Kossin, Anthony Wimmers,

GOES-R Hurricane Intensity Estimation (HIE) Winds-HIE Application Team Chris Velden & Tim Olander (CIMSS) Jaime Daniels (STAR)

The Potential Role of the GPM in Activities at the Naval Research Laboratory Joe Turk and Jeff Hawkins Naval Research Laboratory Marine Meteorology Division.

JOINT TYPHOON WARNING CENTER 2008 YEAR IN REVIEW Mr. Robert (Bob) Falvey Director, JTWC 63 rd INTERDEPARTMENTAL HURRICANE CONFERENCE 2-5 MARCH 2009 ST.

Second IBTrACS Workshop, April 2011, Honolulu, Hawaii 1 ESCAP/WMO Typhoon Committee Best Track Consolidation Meeting, Dec 2010 Hong Kong Summary.

T-PARC2008 Operation Plan of JMA 1Overview of T-PARC2008 activities in JMA 2Outline of Special Observation 3Routine Observation and Special Observation.

2009 TCC presentation JOINT TYPHOON WARNING CENTER 2008 YEAR IN REVIEW Mr. Bob Falvey Director, JTWC Typhoon Sinlaku.

Overview of CIRA and NESDIS Global TC Services Presented by John Knaff NOAA/NESDIS Regional and Mesoscale Meteorology Branch Fort Collins, CO USA For The.

Satellite Data Assimilation Activities at CIMSS for FY2003 Robert M. Aune Advanced Satellite Products Team NOAA/NESDIS/ORA/ARAD Cooperative Institute for.

1 Typhoon Track and Intensity Simulations by WRF with a New TC-Initialization Scheme HIEP VAN NGUYEN and YI-LENG CHEN Department of Meteorology, University.

T-PARC/TCS-08 Quicklook 29 October Operations by the numbers… 9 participating nations –Canada, China, England, France, Germany, Japan, South Korea,

PRELIMINARY VALIDATION OF IAPP MOISTURE RETRIEVALS USING DOE ARM MEASUREMENTS Wayne Feltz, Thomas Achtor, Jun Li and Harold Woolf Cooperative Institute.

J. P. Kossin, 62 nd IHC, Charleston, SC An Objective Tool for Identifying Hurricane Secondary Eyewall Formation Jim Kossin and Matt Sitkowski Cooperative.

1 Current and planned research with data collected during the IFEX/RAINEX missions Robert Rogers NOAA/AOML/Hurricane Research Division.

Munehiko Yamaguchi 12, Takuya Komori 1, Takemasa Miyoshi 13, Masashi Nagata 1 and Tetsuo Nakazawa 4 ( ) 1.Numerical Prediction.

JTWC Satellite Operations (SATOPS) Capt Kathryn Payne 27 April W SINLAKU ATCF.

Satellite Analysis Branch 2009 Year in Review Anthony Salemi with Michael Turk NOAA Hurricane Conference December 1, 2009.

TOWARD AN OBJECTIVE SATELLITE-BASED ALGORITHM TO PROVIDE REAL-TIME ESTIMATES OF TC INTENSITY USING INTEGRATED MULTISPECTRAL (IR AND MW) OBSERVATIONS Jeff.

Tropical Cyclone Forecasting and Monitoring

Rosenstial School of Marine and Atmospheric Science

Tony Reale ATOVS Sounding Products (ITSVC-12)

WP-3D Orion Instrumentation

Defining Uncertainty in Hurricane Maximum Surface Wind Estimation

TC Intensity Estimation: SATellite CONsensus (SATCON)

Medhavy Thankappan and Lan-Wei Wang

UPDATE ON SATELLITE-DERIVED amv RESEARCH AND DEVELOPMENTS

Advanced Dvorak Technique

Objective Dvorak Technique (ODT) AFWA/XOGM

Retrieving the Stratospheric Diurnal Cycle from AMSU measurements

Michael J. Brennan National Hurricane Center

Impact of Assimilating AMSU-A Radiances on forecasts of 2008 Atlantic TCs Initialized with a limited-area EnKF Zhiquan Liu, Craig Schwartz, Chris Snyder,

Andrew Heidinger JPSS Cloud Team Lead

Validation of CIRA Tropical Cyclone Algorithms

Presentation transcript:

Analysis of Satellite-Based TC Intensity Estimation in the WNP Chris Velden and Derrick Herndon University of Wisconsin – Madison Cooperative Institute for Meteorological Satellite Studies (CIMSS) Meteorological Satellite (METSAT) Conference Ford Island Conference Center Pearl Harbor, HI 27-28 April 2009 Research supported by the ONR Marine Meteorology and Atmospheric Effects Program

Objectives and Motivation Analysis of Sat-Based TC Intensity Estimation in the WNP Objectives and Motivation Satellite-based recon is the workhorse for TC monitoring in the WNP, yet the intensity estimation methods have not been carefully validated since a/c recon left the WNP 23 years ago Newly-developed automated methods have become operationally available and show promise, but have only been validated in the Atlantic The TCS-08/TPARC campaigns in 2008 offered a rare opportunity for in situ observations of WNP TC core intensities, and validation of satellite-based estimates Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Assets brought to the WNP for the TCS-08/TPARC field campaigns Analysis of Sat-Based TC Intensity Estimation in the WNP Assets brought to the WNP for the TCS-08/TPARC field campaigns USAF C-130 from the 53rd WRS, with Dropsondes and SFMR NRL P-3, with Dropsondes and Eldora radar Drifting buoys deployed by the C-130 Period of deployment: Early Aug. through early Oct. 2008 Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Satellite-based Methods to be Validated Analysis of Sat-Based TC Intensity Estimation in the WNP Satellite-based Methods to be Validated Dvorak Technique -- IR/VIS, Primary operational tool, Manual Advanced Dvorak Technique (ADT) – Objective/Automated Advanced Microwave Sounding Unit (AMSU) – Obj/Auto, Method based on polar-orbiter 54GHz microwave data SATellite CONsensus (SATCON) – Obj/Auto, Weighted consensus of ADT and AMSU methods Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Validation Experiment Set-up Analysis of Sat-Based TC Intensity Estimation in the WNP Validation Experiment Set-up Automated ADT, AMSU and SATCON all run and documented at CIMSS in real time (CIRA-based AMSU estimates also made available to CIMSS in real time) Real time operational Dvorak estimates collected from JTWC, NESDIS-SAB, and JMA Independently-derived Dvorak estimates by 5 experienced satellite analysts ‘blind’ to the real time recon data and operational Dvorak estimates were made available to CIMSS following the field experiment Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Analysis of Sat-Based TC Intensity Estimation in the WNP Validation Experiment Setup - Additional Notes ADT run in two modes: With and w/o microwave input AMSU estimates from two methods: CIMSS and CIRA (CIRA occasionally unavailable due to method constraints) SATCON reflects availability of 2 or 3 consensus members Dvorak results from JMA include 10min. > 1min. conversion to Vmax, and Koba et al. Tnum > Vmax adjustment Independent-analyst Dvorak estimates provided post-experiment and ‘blind’ to all recon data collected Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

…. cue the crickets

Validation Cases during TCS-08/TPARC Analysis of Sat-Based TC Intensity Estimation in the WNP Validation Cases during TCS-08/TPARC TC Nuri (13W) TC Sinlaku (15W) TC Jangmi (19W) Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Analysis of Sat-Based TC Intensity Estimation in the WNP WC-130J Penetrations Single Double 13W Nuri 15 W Sinlaku 19 W Jangmi 8 Center Fixes Triple 03 Center Fixes 11 Center Fixes TCS-08 aircraft penetration timeline outlining the WC-130J storm penetrations (vortex fixes) accomplished during typhoon Nuri (13W) and typhoon Sinlaku (15 W) during August-September 2008. Typhoon Nuri: two flights on August 17-18 with three (3) penetrations Typhoon Sinlaku: five flights from Sept 9-18, with a total of 11 penetrations Acknowledgments: TCS-08 Management Team Time Line August September 7 14 21 28 1

Analysis of Sat-Based TC Intensity Estimation in the WNP WC-130J storm center fixes within +/- ~4 hours of corresponding AMSU overpasses Storm yyyymmddhhmm lat lon mslp msw amsu_pass(ddhhmm) 13W 200808172300 15.77N 133.62E 994 45 172008 13W 200808182200 16.95N 127.25E 977 78 182034 15W 200809090600 17.87N 125.25E 986 62 090511 15W 200809100600 20.24N 124.33E 954 90 100501 15W 200809100800 20.42N 124.37E 946 100 100807 15W 200809111300 21.80N 124.75E 940 90 110819 15W 200809121700 23.83N 123.22E 953 90 121713 15W 200809180400 30.33N 130.24E 981 65 180818 15W 200809190400 33.02N 135.09E 975 75 190755 15W 200809191800 34.18N 139.22E 978 65 192014 19W 200809242100 13.50N 134.18E 991 55 242001 19W 200809260000 15.77N 129.65E 973 75 251640 19W 200809260200 16.10N 129.35E 967 80 260506 19W 200809270900 21.09N 124.78E 904 135 270832 List of TCS-08 storms/flights with collocated AMSU overpasses (light blue slightly exceed +/- 3 hour time window). These cases do NOT include MetOp AMSU cases since this sensor is not yet incorporated into the TC intensity processing data stream. Acknowledgments: Derrick Herndon TCS-08 satellite validation cases were limited!

Analysis of Sat-Based TC Intensity Estimation in the WNP The final values for Max Sustained Wind (MSW or Vmax) used for validation of the satellite-based estimates were derived from multiple recon sources for each eye penetration by a selected team of analysts headed by Prof. R. Elsberry (NPS) Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Analysis of Sat-Based TC Intensity Estimation in the WNP Preliminary Findings None of the results are statistically significant due to the very limited number of validation cases Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Analysis of Sat-Based TC Intensity Estimation in the WNP Comparison of Dvorak-Based Estimates – Vmax (Kts) N=15 ‘Blind’ Dvorak Consensus Oper (w/JMA Koba adj) ADT w/MW Bias 5.0 4.1 3.6 0.0 -1.1 Abs Error 10.9 15.0 13.0 16.1 14.1 RMSE 14.0 18.0 15.2 19.4 17.6 Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process) Positive Bias indicates method estimates are too strong

Analysis of Sat-Based TC Intensity Estimation in the WNP Comparison of “Blind” Dvorak Analyst Estimates – Vmax (Kts) N=15 Analyst 1 2 3 4 5 Bias 1.7 8.1 9.9 2.0 1.2 Abs Error 7.9 11.9 16.9 12.7 12.6 RMSE 9.6 15.8 19.9 15.7 15.0 Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process) Positive Bias indicates method estimates are too strong

Analysis of Sat-Based TC Intensity Estimation in the WNP Comparison of All Satellite-based Estimates – Vmax (Kts) N=13 ‘Blind’ Dvorak Consensus Oper (w/Koba) ADT w/MW CIMSSAMSU SATCON Bias 2.9 1.4 -5.8 3.1 0.2 Abs Error 9.1 12.3 12.8 9.2 RMSE 11.8 14.8 16.6 10.7 11.1 Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process) Positive Bias indicates method estimates are too strong

Typhoon Nuri (13W) Intensity Estimates Versus Recon } Objective Methods Subjective Methods } Typhoon Nuri Recon Recon mission truncated

Analysis of Sat-Based TC Intensity Estimation in the WNP Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Analysis of Sat-Based TC Intensity Estimation in the WNP

Analysis of Sat-Based TC Intensity Estimation in the WNP

Analysis of Sat-Based TC Intensity Estimation in the WNP Recon Objective Methods Subjective Methods } Typhoon Jangmi Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Summary of Preliminary Findings Analysis of Sat-Based TC Intensity Estimation in the WNP Summary of Preliminary Findings (Based on limited sample of 15 recon validation points) Ave. Vmax estimate errors (kts): Dvorak: ~11 (blind), ~13 (oper), ~14 (ADT) Subj Dvorak ave error spread (kts): 8-17 (‘blind’ analysts), 11-15 (oper agencies) [JMA (incl their Koba et al. Tnum>Vmax adjustment) superior to other 2 agencies] AMSU (CIMSS) and SATCON ave errors (kts): Both ~ 9 (subset of 13 val. pts) General Preliminary Conclusions Objective satellite- based methods are very competitive with Dvorak Significant spread in subjective Dvorak estimates Consensus means improve accuracies for all methods Need additional validation points for statistical confidence in results Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Analysis of Sat-Based TC Intensity Estimation in the WNP Acknowledgements ONR Marine Meteorology and Atmospheric Effects Program under the direction of R. Ferek and D. Eleuterio for supporting TCS-08 and aircraft assets USAF 17 OWS/CC (Lt. Col. K. Brueske) for help in getting the C-130 to the WNP theater USAF 53rd WRS and the NRL P-3 crews for providing the critical validation observations JTWC (R. Falvey and K. Payne) and NRL-MRY (J. Hawkins) for their support of this exercise and the satellite R&D effort Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Analysis of Sat-Based TC Intensity Estimation in the WNP Acknowledgements Independent Dvorak Analysts J. Beven and D. Roberts (NHC) A. Burton (Australian BoM) P. McCrone (FNMOC) K. Viault (JTWC) S. Nishimura (JMA) Vmax Validation Point Analysts J. Hawkins (NRL-MRY) P. Black (NRL-MRY) E. Sanabia (NPS) J. Beven (NHC) Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)

Analysis of Sat-Based TC Intensity Estimation in the WNP “Mai Tai’s taste a lot better than Miller Lites …wish I was there!!” Chris Velden (from his post-op recovery pad in Madison) Notes: Verification Process Status and Issues: (How well does the product/software fulfill its functional requirement or intended use – AMOP VTR Status and Issues) Validation Process Status and Issues: (How well does the product/system/hardware “as built” fulfill its intended Operational performance goal - Operational Test Status and Issues) Measures of Performance: (principal metrics or test criteria used or plan to use in the V&V Process)