WLTP gtr Annex 9 Determination of Method Equivalency - Progress Report WLTP-11 23e WLTP gtr Annex 9 Determination of Method Equivalency - Progress Report K. Kolesa, 2.6.2015

Situation Conclusion of meeting WLTP #10: Proposal for determination of system equivalence will be developed based on provisions of gtr 4 and US Regulation 1065 . It was requested to use definitions of ISO 5725 Proposal is expected to be presented in WLTP IWG meeting #11. Meanwhile results: Experts have reviewed the ISO Standard 5725. The standard is complete but very specific and complex. General requirements can be applied. Details need a thorough statistical background. Experts discovered slightly different statistical formulae in gtr 4 and US Regulation 1065. General approach for Annex 9 was confirmed, but further work and validation on implementation prescription is needed K. Kolesa, May 2015

Definitions in ISO 5725-1 K. Kolesa, May 2015

Basic Recommendations in ISO 5725-6 for comparison of alternative measurement methods (I) Requirements for international standard measurement methods: Applicable to a wide range of levels of characteristics Equipment shall be available on an international basis Cost of performing the measurement shall be acceptable The precision and trueness of the measurement methods shall be acceptable for the users of the results. The comparison between methods shall be made on the results of precision experiments on reference and candidate method. Both experiments shall be conducted in accordance with ISO-5725-2 The experimental results should be able to detect if the ratio between the precisions of the compared methods are in a specified range The trueness of both method shall be proven equal involving identical samples by showing that the difference in mean values is statistically insignificant (e.g. 95 % confidence level) The difference between the expected values of the results of the compared methods, if available, shall be in a specified range K. Kolesa, May 2015

Basic Recommendations in ISO 5725-6 for comparison of alternative measurement methods (II) The accuracy experiment shall be conducted in accordance with ISO 5725-1 The procedures for both methods shall be documented in sufficient detail so as to avoid misinterpretations. No modification to the procedure is permitted during the experiment. As precision of measurement methods is affected by the matrix of test samples comparison is best done on identical test samples. Required number of test samples varies depending on the range of the characteristic levels of interest and the dependency of the accuracy on the level. The candidate measurement method shall provide the same or better precision under repeatable conditions K. Kolesa, May 2015

Proposal for Inclusion in GTR 15 (I) Annex 9 Determination of method equivalency 1. General Requirement Upon request of the manufacturer other measurement methods may be approved by the type approval authority, if it is found that they yield equivalent results in accordance with paragraph 1.1. The equivalence of the candidate method shall be demonstrated to and accepted by the authority. 1.1 Decision on Equivalency A candidate method shall be considered equivalent, if the accuracy and the precision is same or better than the reference method. 1.2 Determination of Equivalency The determination of system equivalency shall be based on a correlation study between the candidate and the reference methods. The methods to be used for correlation testing shall be subject to the approval by the authority. The basic principle for determination of accuracy and precision of candidate and reference methods shall applied following the guidance laid down in ISO 5725 Part 6 Annex 8 “Comparison of alternative Measurement Methods”. 1.3 Implementation requirements (reserved) K. Kolesa, May 2015

Implementation Approach of Heinz Steven Determination of System equivalency: Perform ncand measurements with the candidate system and nref measurements with the reference system and compare standard deviations and averages with the F-test and the t-test. ncand and nref need to be equal or higher than 7 and ideally ncand = nref. 1.step: Check, whether the hypothesis σcand = σref is accepted or rejected. The counterhypothesis is σcand > σref, in which case the candidate method is not equivalent. If scand²/sref² <= F1-α(fcand; fref), then sigmacand = sigmaref is not rejected, then step 2 has to be applied. If it is rejected, the candidate system is not equivalent. fcand = ncand – 1, fref = nref – 1, ncand and nref are the sample sizes. α = 10%. 2. step: Check, whether the hypothesis µcand = µref is accepted or rejected. The counterhypotheses are µcand > µref and µcand < µref. If Abs(xave, cand - xave, ref)*Sqrt(ncand*nref/(ncand +nref))/s <= tf; 1- α, then µcand = µref is not rejected and the candidate system is equivalent to the reference system. If it is rejected, the candidate system is not equivalent. f = ncand + nref – 2, α = 10%. s = Sqrt(((ncand – 1)*scand² + (nref – 1)*sref²)/(ncand + nref – 2)) This is equivalent to the text in annex 4 of GTR #4, except for the first step, where it says in annex 4 of GTR #4, but it needs to read K. Kolesa, May 2015

Conclusion and Proposal for further Proceeding IWG WLTP is informed about the on going work of alignment of gtr 4 and 1065 provisions with SAE 5725. For WLTP phase 1b it is proposed to include the drafted text for Annex 9 In WLTP phase 2 the proposed implementation prescription should be validated by an expert group with statistical background to show effects for comparison of equipment and procedures IWG WLTP is asked to support this two step approach and to accept proposed text for inclusion in gtr 15 K. Kolesa, May 2015