1. WLTP gtr Annex 9 Determination of Method Equivalency - Progress Report
WLTP-11 23e
WLTP gtr Annex 9 Determination of Method Equivalency - Progress Report
K. Kolesa,

2. Situation Conclusion of meeting WLTP #10:
Proposal for determination of system equivalence will be developed based on provisions of gtr 4 and US Regulation
It was requested to use definitions of ISO 5725
Proposal is expected to be presented in WLTP IWG meeting #11.
Meanwhile results:
Experts have reviewed the ISO Standard The standard is complete but very specific and complex.
General requirements can be applied. Details need a thorough statistical background.
Experts discovered slightly different statistical formulae in gtr 4 and US Regulation 1065.
General approach for Annex 9 was confirmed, but further work and validation on implementation prescription is needed
K. Kolesa, May 2015

3. Definitions in ISO
K. Kolesa, May 2015

4. Basic Recommendations in ISO 5725-6 for comparison of alternative measurement methods (I)
Requirements for international standard measurement methods:
Applicable to a wide range of levels of characteristics
Equipment shall be available on an international basis
Cost of performing the measurement shall be acceptable
The precision and trueness of the measurement methods shall be acceptable for the users of the results.
The comparison between methods shall be made on the results of precision experiments on reference and candidate method. Both experiments shall be conducted in accordance with ISO
The experimental results should be able to detect if the ratio between the precisions of the compared methods are in a specified range
The trueness of both method shall be proven equal involving identical samples by showing that the difference in mean values is statistically insignificant (e.g. 95 % confidence level)
The difference between the expected values of the results of the compared methods, if available, shall be in a specified range
K. Kolesa, May 2015

5. Basic Recommendations in ISO 5725-6 for comparison of alternative measurement methods (II)
The accuracy experiment shall be conducted in accordance with ISO
The procedures for both methods shall be documented in sufficient detail so as to avoid misinterpretations. No modification to the procedure is permitted during the experiment.
As precision of measurement methods is affected by the matrix of test samples comparison is best done on identical test samples.
Required number of test samples varies depending on the range of the characteristic levels of interest and the dependency of the accuracy on the level.
The candidate measurement method shall provide the same or better precision under repeatable conditions
K. Kolesa, May 2015

6. Proposal for Inclusion in GTR 15 (I)
Annex 9 Determination of method equivalency
1. General Requirement
Upon request of the manufacturer other measurement methods may be approved by the type approval authority, if it is found that they yield equivalent results in accordance with paragraph 1.1. The equivalence of the candidate method shall be demonstrated to and accepted by the authority.
1.1 Decision on Equivalency
A candidate method shall be considered equivalent, if the accuracy and the precision is same or better than the reference method.
1.2 Determination of Equivalency
The determination of system equivalency shall be based on a correlation study between the candidate and the reference methods. The methods to be used for correlation testing shall be subject to the approval by the authority.
The basic principle for determination of accuracy and precision of candidate and reference methods shall applied following the guidance laid down in ISO 5725 Part 6 Annex 8 “Comparison of alternative Measurement Methods”.
1.3 Implementation requirements (reserved)
K. Kolesa, May 2015

7. Implementation Approach of Heinz Steven
Determination of System equivalency:
Perform ncand measurements with the candidate system and nref measurements with the reference system and compare standard deviations and averages with the F-test and the t-test. ncand and nref need to be equal or higher than 7 and ideally ncand = nref.
1.step: Check, whether the hypothesis σcand = σref is accepted or rejected. The counterhypothesis is σcand > σref, in which case the candidate method is not equivalent.
If scand²/sref² <= F1-α(fcand; fref), then sigmacand = sigmaref is not rejected, then step 2 has to be applied. If it is rejected, the candidate system is not equivalent. fcand = ncand – 1, fref = nref – 1, ncand and nref are the sample sizes. α = 10%.
2. step: Check, whether the hypothesis µcand = µref is accepted or rejected. The counterhypotheses are µcand > µref and µcand < µref.
If Abs(xave, cand - xave, ref)*Sqrt(ncand*nref/(ncand +nref))/s <= tf; 1- α, then µcand = µref is not rejected and the candidate system is equivalent to the reference system. If it is rejected, the candidate system is not equivalent.
f = ncand + nref – 2, α = 10%.
s = Sqrt(((ncand – 1)*scand² + (nref – 1)*sref²)/(ncand + nref – 2))
This is equivalent to the text in annex 4 of GTR #4, except for the first step, where it says
in annex 4 of GTR #4, but it needs to read
K. Kolesa, May 2015

8. Conclusion and Proposal for further Proceeding
IWG WLTP is informed about the on going work of alignment of gtr 4 and 1065 provisions with SAE 5725.
For WLTP phase 1b it is proposed to include the drafted text for Annex 9
In WLTP phase 2 the proposed implementation prescription should be validated by an expert group with statistical background to show effects for comparison of equipment and procedures
IWG WLTP is asked to support this two step approach and to accept proposed text for inclusion in gtr 15
K. Kolesa, May 2015