Marian Muste1 Juan Gonzalez-Castro2 Dongsu Kim1 Kwonkyu Yu1 Uncertainty Analysis for Flow Measurements and Techniques using Standardized Methodology Marian Muste1 Juan Gonzalez-Castro2 Dongsu Kim1 Kwonkyu Yu1 Thank you for inviting me to present a part of my work. As I am close to my stay in Karlsruhe, I would like to thank to the people of the Institute for rewarding me with a memorable experience. Thank Prof. Jirka and Rodi for inviting me and as well to Volker, Tobias, Thorsten, Connie, Elleta, Charlotte, Gregor, Andreas, Gerd for interacting closer during my stay here. Above all, thank you for the wonderful coffee breaks, that I found essential for the progress in research. Back at IIHR I work well, but here I work well and with pleasure. I hope to become a frequent back-commer here in Karlsruhe, as some of my predecessors are these days. 1 IIHR- Hydroscience & Engineering, The University of Iowa 2 South-Florida Water Management District, West Palm Beach

Overview Background Uncertainty Analysis (UA) Frameworks AIAA (1995) UA Implementation Example Methodology Assessment of Elemental Uncertainties Customized GUI for UA Implementation Conclusions Outlook

regarding uncertainty analysis originated from a Hydrologic Service Background SAMPLE REQUEST regarding uncertainty analysis originated from a Hydrologic Service …. Has anybody out there had to defend the validity of an ADCP flow measurement against a legal challenge from a third party? ……When current meters were used to undertake these measurements we could claim that the flow measurement was undertaken in conformance with British and International standards for current meter gauging and that the current meter had a valid calibration certificate… In the case where flow measurements are now taken using ADCPs we feel more vulnerable to legal challenges. This is for two reasons: 1. There is no ISO document in place. The Agency has to rely on its own internal document on gauging procedures which is based on the draft ISO document. 2. ADCPs do not have "certificates of calibration" . The only checks on the performance that can be made are against other ADCPs or other types of flow monitoring equipment. (posted on the USGS’ Hydro-Acoustics Work Group webpage by R. Iredale, The Environment Agency of England and Wales, 2005)

Guide to the Expression of Uncertainty Measurement (ISO, 1993) UA Frameworks Over the last 50 years, considerable efforts have been made by professional societies to develop and implement uncertainty analysis (UA). One of the rigorous UA methodology (based on sound statistical and engineering concepts): Guide to the Expression of Uncertainty Measurement (ISO, 1993) - adopted widely by various scientific & research communities, e.g., NIST (1994), NF ENV 13005 (1999) - the guide recognizes the need for further adaptation for specific areas Specific adaptations for engineering: - Assessment of Wind Tunnel Data Uncertainty (AIAA, 1995) - Test Uncertainty (ASME, 1998) Key assumptions/concepts for ISO (1993)-based standards - Gaussian pdf-s for the error sources - 2 sample standard deviations for 95% confidence level - for large samples (N ≥ 10), special procedures for handling small samples - RSS used for combining uncertainties - Taylor-series expansion for propagation of uncertainties - total uncertainties expressed using confidence intervals

UA Frameworks Terminology for ISO (1993) - based standards The 3 standards provide the same total measurement uncertainty

Engineering approach, simple, clear, widely applied AIAA (1995) Engineering approach, simple, clear, widely applied Bias error (b): fixed, systematic Bias limit (B): estimate of b Precision error (e): random Precision limit (P): estimate of e Total error: d = b + e

Implementation Sequence AIAA (1995) Implementation Sequence Key feature: data-reduction equation r = r(X1, X2, X3,…, Xj)

Implementation Aspects AIAA (1995) Implementation Aspects Measurement systems for each individual variable Xi : instrument, data acquisition and reduction procedures, operational environment (laboratory, in situ), the flow and its interaction with the instrument and the environment Estimates of errors are meaningful only when considered in the context of the process leading to the value of the quantity under consideration Uncertainties estimated following the signal propagation from sensor to the final result Uncertainties estimated with a pre-established confidence level (95% for most engineering areas) UA differently conducted dependent on the type of experiment: Single test (for complex or expensive experiments): one set of measurements (X1, X2, …, Xj) for r Multiple tests (ideal situations): many sets of measurements (X1, X2, …, Xj) for r at a fixed test condition with the same measurement system

MULTIPLE TESTS (recommended) AIAA (1995) MULTIPLE TESTS (recommended) Given a data reduction equation for a measurement The result and its uncertainty is with The uncertainty in the final result where the bias limit of the result is and the precision limit of the result is

AIAA (1995) SINGLE TEST Given a data reduction equation for a measurement The result and its uncertainty is The uncertainty in the final result where the bias limit of the result is and the precision limit of the result is  Based on prior information

Implementation Aspects AIAA (1995) Implementation Aspects sound engineering judgment to optimize the output with minimum costs, e.g.: use of end-to-end uncertainty estimation approach uncertainty sources < 1/4 or 1/5 of the largest sources are usually considered negligible specific procedures for single and multiple measurements specific procedures for dealing with small statistical samples methodology for assessment of calibration uncertainties methodology for data validation

Implementation Aspects AIAA (1995) Implementation Aspects Integration of UA in all phases of the measurement

IMPLEMENTATION EXAMPLE AIAA (1995) IMPLEMENTATION EXAMPLE Extensively used in laboratory measurements and field conditions, from simple (Pitot tube) to complex (LDV) instruments Widely applied for teaching and research purposes Successful implementation to discharge measurements: conventional instruments (Muste et al. 2007) contemporary, nonintrusive techniques: Large-Scale Particle Image Velocimetry (Y-S. Kim et al, 2007) Acoustic-Doppler Current Profilers (Gonzalez-Castro & Muste, 2007)

ADCP UA: Implementation Currently, ADCPs are the most efficient instrument for riverine environment characterization (monitoring and research needs) If properly operated, the instrument can accurately document discharges, mean velocities, and selected turbulence characteristics Despite their extensive use, there are aspects regarding their capabilities, operation, and uncertainty analysis not documented yet

ADCP Uncertainty Analysis (UA) status ADCP UA: Implementation ADCP Uncertainty Analysis (UA) status Past efforts (non-standardized methodologies) Discharge: Simpson & Oltman (1992), Gordon (1993), Lipscomb (1995), Morlock (1996), Simpson (2001), Gartner (2002), Muller (2002), Yorke & Oberg (2002), USGS-RDI (2005) Turbulence measurements: Droz (1998), Stacey (1999), Nystrom (2002), Schemper & Admiraal (2002) On-going efforts (standardized methodology) UA formulated within the framework of authoritative engineering standards

ADCP UA: Implementation Discharge Measurement with ADCP mounted on a boat where

ADCP UA: Elemental Uncertainty Assessment Error identification

ADCP UA: Implementation Data Reduction Equations (Teledyne/RDI’s ADCP)

ADCP UA: Implementation Exact approach – discharge in the direct measured area Using BT

ADCP UA: Implementation Exact approach: in-bin discharge Using BT If , the discharge is a functional relationship of the form:

ADCP UA: Implementation Exact approach – top and bottom discharges (extrapolation)

ADCP UA: Implementation Uncertainty Propagation to Final Result: Bias Limit

ADCP UA: Implementation Uncertainty Propagation to Final Result: Precision Limit Uncertainty Propagation to Final Result: Total Uncertainty

ADCP UA: Implementation Practical approach (pitch and roll neglected in DRE; errors accounted through end-to-end calibrations) velocity (instrument coordinates neglecting the pitch and roll angle) = beam angle, = angle of the flow to instrument β = angle of the boat velocity = in beam water velocities = boat velocity where total discharge

ADCP UA Software - architecture Developing tools - Borland C++ Builder (v.6) & Microsoft Access Software Configuration

ADCP UA Software - GUIs Archive database - Elemental uncertainties are archived in categories based on river characteristics. - Users with limited level of preparedness can estimate uncertainties using default values obtained in similar environment and operating conditions. - The stored information is updated as soon as new measurements are processed. - User can also create new archives using new classification categories

ADCP UA Software - GUIs Information for archiving

ADCP UA Software - GUIs Assessment of bias limit

ADCP UA Software - GUIs Assessment of precision limit

ADCP UA Software & GUIs Assessment of total uncertainty

Conclusions Feasibility of UA engineering standards for implementation to ADCP measurements The methodology is comprehensive, simple to implement Easily upgradeable as new info occur UA allows tracing of the measurement accuracy to primary standards  withstand legal and strict QA/QC requirements Finalization of UA – an extensive and expensive effort Collaboration between manufacturers and users in a coordinated effort = key to complete UA for the variety of measurement situations and operating conditions encountered in monitoring practice The framework was adopted by ASCE’s HME Task Committee and the UNESCO group on Data Requirements for Integrated Urban Water Management (Fletcher et al., 2007) Currently evaluated by the ISO committee (Herschy) - The capabilities and limitations are presented throughout the lectures in connection with the issues discussions Assumption made herein is that the basic operating parameters selected during data collection (e.g., bin size, mode, bottom track, water pings per ensemble, blanking distance) adequately match physical conditions such that we have mainly to deal with inherent errors and flow characteristics 8

Conclusions The UA customized software for ADCP velocity and discharge measurements requires minimum user preparation Autoarchiving uncertainties for specific environments and operating conditions can provide information about dominant sources of uncertainties at various sites. By continuously increasing the sample size through archiving, the UA output is progressively enhanced.

Outlook Work closely with manufacturers and users to assess elemental error sources (manufacturer, operator, environment, or combinations) and integrate them in the AIAA (1995) uncertainty assessment framework for rigorous documenting velocity and discharge measurement accuracy Conduct sensitivity analysis and field tests for compiling uncertainty minimizations guidelines Develop operational guidelines for conducting accurate measurements in various flow regimes

Outlook Need for coordination and extensive collaboration among ADCP manufacturer, operators, data users, and third-party evaluators Need for evaluation of the status of current developments and to strategize for integrative efforts to assess methodologies for operation and accuracy assessment of the ADCP as well as other flow measurement techniques over an extend the range of flow conditions (present WMO effort) IIHR is willing to be actively involved in the WMO initiative

Thank you! Questions?