WLTP CoP Procedure for CO2/FC Analysis of CoP data and JRC Proposal assessment B. Ciuffo, A. Marotta, M. Ktistakis WLTP CoP TF telco June 24, 2019

Conformity of production (CoP) (background info) CoP is a means of evidencing the ability to produce a series of products that comply with the specifications, performance and marking requirements demonstrated and outlined in the type approval documentation. CoP should thus be sufficiently robust to ensure the conformity of what is produced with what has been demonstrated during type- approval Given the complexity of the “vehicle” product, CoP is not a trivial task as there are many uncertainties In the production process In the testing method

Conformity of production (CoP) (background info) Managing the existing uncertainty is a task that should not be underestimated It is necessary to ensure sufficient evidence to base the conformity upon It is important to follow sound statistical procedures in order to have sufficient confidence on the results of the assessment In particular, what we want to achieve at the end of the CoP is sufficient confidence that there is no significant difference between the emissions and CO2/FC measured during the type- approval process (thus representative of the entire population) and what is measured from the population sample

Conformity of production (CoP). Conditions (background info) Population. During the emission type approval, CO2 results are compared with a declared value (DV). DV is accepted if the resulting CO2 is lower than DV When it comes to CoP, there are 3 reasons why DV should be higher than the population mean: Customer protection (the CO2/FC declared on the CoC should be higher than the individual vehicle CO2/FC value under WLTP, for the majority of the consumers); Imposing a margin between DV and population mean creates an incentive at production level to improve the quality and decrease the CO2/FC spread; Assuming that DV could be equal to the population mean would penalise manufacturers with higher production quality standards. Confidence Interval. In the hypothesis that the different CoP samples are uncorrelated and that the production process does not change over time, confidence intervals can be the right tool to assess whether the population is in line with the expectation with a certain confidence Confidence interval around the mean with 90% or 95% confidence can be used

Conformity of production (CoP). Approach (background info) In order to use a pragmatic approach, 16 can still be the maximum sample size The boundaries of the confidence interval are calculated using a Student distribution with (N-1) degrees of freedom and the standard deviation of the sample DV is assumed to be the upper bound of the acceptance region in case of 16 tests The assumption on DV leads to the identification of the theoretical mean of the CO2 distribution. This depends on the confidence level and the standard deviation of the distribution (either the theoretical value or the sample value)

Conformity of production (CoP). Acceptance-Rejection Region (s=3%) (background info) m=DV-(t90,(16-1)*s/√16) DV m

Assessment of the method (1) (background info) The method was tested using a simulation approach. In particular 100.000 combinations of 16 randomly generated results of CoP tests have been derived from a population following a Normal distribution having mean (m) standard deviation (s) Simulation results derived in terms of Probability for a vehicle to have CO2 higher than the declared value (Pr{x<DV}): Defective rate Overall Pass/Fail probability Pass/Fail rate at the end of each CoP step

Assessment of the method (2) The method was also tested using the CoP data provided by France, Germany and The Netherlands. In particular 100.000 combinations of 16 results of CoP tests have been randomly sampled from the available data Simulation results derived in terms of Overall Pass/Fail probability Pass/Fail rate at the end of each CoP step Before applying the method, the three datasets were analysed and centred around a common mean value to remove the variability due to the selection of the declared value for the specific family

Assessment of the method (2.1) During June 6 telco it was requested to JRC to make additional analyses: Centering data around additional values (not just 100, 99, and 98) See the effect of different max number of tests on the pass/fail results See the results of applying the current method on the same data

Histograms of data cantered around 100% Overview of CoP data distributions Histograms of raw data Histograms of data cantered around 100%

Number of families which have 16,17,…,172 vehicles French dataset. Preprocessing We use 611 vehicles from the 11 families which have at least 16 vehicles Number of families which have 16,17,…,172 vehicles 16 21 25 27 37 44 58 83 96 172 2 1 Raw data Data shifted to 100% min 1st Qu median mean 3rd Qu max sd 83.40 92.50 94.70 94.60 96.80 102.50 3.10 min 1st Qu median mean 3rd Qu max sd 88.34 98.66 100.07 100.00 101.48 107.28 2.31

French dataset. Assessment of JRC method Raw data N. tests Pass Rate (%) Fail Rate 3 46.71 0.00 4 20.92 Cumulative 67.63 5 13.36 6 8.05 7 4.85 8 2.85 96.75 9 1.52 10 0.85 11 0.41 12 0.26 99.78 13 0.12 14 0.05 15 0.02 16 Total 100.00 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 67.66 0.00 4 18.13 Cumulative 85.79 5 7.64 6 3.37 7 1.80 8 0.82 99.42 9 0.34 10 0.14 11 0.06 12 0.02 99.99 13 0.01 14 15 16 Total 100.00 Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 52.38 0.00 4 20.63 Cumulative 73.02 5 10.41 6 6.22 7 3.83 8 2.40 95.88 9 1.61 10 1.01 11 0.58 12 0.36 99.45 13 0.24 14 0.13 15 0.08 16 0.10 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 35.42 0.02 4 16.85 0.01 Cumulative 52.27 5 11.80 6 8.19 0.00 7 5.79 8 4.66 82.71 0.03 9 3.49 10 2.78 11 2.22 12 1.80 92.99 13 1.47 14 1.17 15 0.97 16 3.37 Total 99.97

French dataset. Assessment of JRC method Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 19.81 0.29 4 9.83 0.11 Cumulative 29.64 0.40 5 7.20 0.06 6 5.87 0.04 7 4.73 0.03 8 4.29 0.02 51.74 0.55 9 3.62 0.01 10 3.31 11 2.96 12 2.56 64.19 0.59 13 2.33 0.00 14 2.23 15 2.03 16 28.62 Total 99.41 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 8.54 1.99 4 3.65 1.29 Cumulative 12.18 3.28 5 2.46 1.15 6 1.80 1.02 7 1.44 0.88 8 1.20 0.71 19.08 7.04 9 0.65 10 0.94 0.62 11 0.81 0.51 12 0.68 0.49 22.53 9.31 13 0.59 0.45 14 0.57 0.39 15 0.55 16 64.84 0.38 Total 89.08 10.92 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.39 9.10 4 0.79 7.30 Cumulative 3.18 16.4 5 0.40 6.75 6 0.21 6.01 7 0.16 5.30 8 0.09 4.83 4.04 39.29 9 0.07 4.13 10 0.04 3.82 11 0.03 3.41 12 0.01 3.11 4.19 53.75 13 0.02 2.92 14 2.75 15 2.53 16 31.47 2.35 Total 35.69 64.31 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.50 24.69 4 0.11 18.69 Cumulative 0.61 43.38 5 0.03 14.15 6 0.01 10.24 7 0.00 7.48 8 5.70 0.66 80.95 9 4.27 10 3.14 11 2.56 12 1.96 92.87 13 1.56 14 1.16 15 0.89 16 2.17 0.69 Total 2.83 97.17

Assessment of JRC method 𝑙𝑏 𝑖 =1− 𝑡 𝑖−1,𝑐𝐿 𝑖 + 𝑡 𝑁−1,𝑐𝐿 𝑁 ∗𝑠 𝑖 𝑢𝑏(𝑖)=1+ 𝑡 𝑖−1,𝑐𝐿 𝑖 − 𝑡 𝑁−1,𝑐𝐿 𝑁 ∗𝑠(𝑖)

French dataset. Assessment of JRC method Raw data N. tests Pass Rate (%) Fail Rate 3 44.24 0.00 4 19.80 Cumulative 64.03 5 13.40 6 8.40 7 5.56 8 3.35 94.74 9 2.16 10 1.31 11 0.77 12 1.02 Total 100.00 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 64.19 0.00 4 18.56 Cumulative 82.75 5 8.10 6 4.40 7 2.43 8 1.15 98.83 9 0.62 10 0.29 11 0.12 12 0.14 Total 100.00 Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 49.47 0.00 4 20.71 Cumulative 70.18 5 10.68 6 6.59 7 4.16 8 2.71 94.32 9 2.13 10 1.27 11 0.73 12 1.55 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 34.72 0.01 4 15.02 Cumulative 49.74 5 10.81 0.02 6 7.95 7 5.83 8 4.70 79.03 0.04 9 3.98 10 2.62 11 2.52 12 11.81 Total 99.96

French dataset. Assessment of JRC method Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 18.21 0.38 4 8.44 0.11 Cumulative 26.65 0.49 5 6.57 6 5.03 0.08 7 4.31 0.05 8 3.79 0.03 46.35 0.76 9 3.33 0.06 10 2.96 0.02 11 2.76 12 43.73 0.01 Total 99.13 0.87 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 7.49 2.14 4 3.14 1.89 Cumulative 10.63 4.03 5 1.90 1.80 6 1.38 1.44 7 1.21 1.08 8 1.15 1.00 16.27 9.35 9 0.90 1.07 10 0.62 0.99 11 0.61 0.96 12 68.45 0.78 Total 86.85 13.15 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.33 9.96 4 0.71 8.72 Cumulative 3.04 18.68 5 0.39 7.56 6 0.19 7.35 7 0.10 6.33 8 0.03 5.39 3.75 45.31 9 0.08 4.84 10 0.00 4.24 11 0.02 4.15 12 34.18 3.43 Total 38.03 61.97 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.41 26.86 4 0.09 21.04 Cumulative 0.50 47.90 5 0.02 14.89 6 0.01 10.29 7 0.00 7.41 8 5.07 0.53 85.56 9 4.06 10 2.58 11 2.04 12 3.75 1.48 Total 4.28 95.72

French dataset. Assessment of JRC method Raw data N. tests Pass Rate (%) Fail Rate 3 40.50 0.00 4 17.63 Cumulative 58.13 5 12.02 6 8.60 7 6.04 8 15.21 Total 100.00 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 60.74 0.00 4 18.78 Cumulative 79.52 5 8.38 6 4.99 7 2.77 8 4.33 Total 100.00 Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 45.85 0.00 4 18.41 Cumulative 64.26 5 10.88 6 6.70 7 4.62 8 13.54 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 30.34 0.03 4 13.51 0.02 Cumulative 43.85 0.04 5 9.64 0.01 6 7.36 0.00 7 5.38 8 33.71 Total 99.94 0.06

French dataset. Assessment of JRC method Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 16.76 0.35 4 7.23 0.30 Cumulative 23.99 0.65 5 4.94 0.21 6 3.69 0.24 7 3.08 0.18 8 62.89 0.13 Total 98.59 1.41 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 6.87 2.74 4 2.51 2.67 Cumulative 9.38 5.41 5 1.56 2.73 6 0.98 7 0.77 2.57 8 71.72 2.36 Total 84.41 15.59 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.06 11.82 4 0.50 11.85 Cumulative 2.56 23.67 5 0.25 11.29 6 0.13 9.49 7 0.06 7.89 8 38.28 6.39 Total 41.27 58.73 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.41 32.16 4 0.06 23.90 Cumulative 0.47 56.07 5 0.01 15.62 6 0.00 9.71 7 6.38 8 7.63 4.10 Total 8.12 91.88

French dataset. Assessment of JRC method Cumulative Ν=16 Ν=12 Ν=8 Raw data 4 67.63 64.03 58.13 8 96.75 94.74 100.00 12 99.78 - 16 95% centered data 85.79 82.75 79.52 99.42 98.83 99.99 96% centered data 73.02 70.18 64.26 95.88 94.32 99.45 97% centered data 52.27 49.74 43.85 82.71 79.03 99.94 92.99 99.96 99.97 Cumulative Ν=16 Ν=12 Ν=8 98% centered data 4 29.64 26.65 23.99 8 51.74 46.35 98.59 12 64.19 99.13 - 16 99.41 99% centered data 12.18 10.63 9.38 19.08 16.27 84.41 22.53 86.85 89.08 100% centered data 3.18 3.04 2.56 4.04 3.75 41.27 4.19 38.03 35.69 101% centered data 0.61 0.50 0.47 0.66 0.53 8.12 4.28 2.83

French dataset. Comparison with current CoP method

French dataset. Assessment of Current method Raw data N. tests Pass Rate (%) Fail Rate 3 100.00 0.00 4 Cumulative 5 6 7 8 9 10 11 12 13 14 15 16 Total Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 100.00 0.00 4 Cumulative 5 6 7 8 9 10 11 12 13 14 15 16 Total Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 99.99 0.01 4 0.00 Cumulative 5 6 7 8 9 10 11 12 13 14 15 16 Total Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 99.85 0.12 4 0.03 Cumulative 99.88 5 0.00 6 7 8 9 10 11 12 13 14 15 16 Total

French dataset. Assessment of Current method Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 99.04 0.85 4 0.10 0.01 Cumulative 99.14 0.86 5 0.00 6 7 8 9 10 11 12 13 14 15 16 Total Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 93.74 5.78 4 0.42 0.04 Cumulative 94.16 5.83 5 0.01 6 0.00 7 8 94.17 9 10 11 12 13 14 15 16 Total Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 77.04 21.91 4 0.69 0.32 Cumulative 77.72 22.23 5 0.03 0.01 6 7 0.00 8 77.76 22.24 9 10 11 12 13 14 15 16 Total Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 47.06 51.63 4 0.60 0.63 Cumulative 47.66 52.26 5 0.02 0.05 6 0.01 7 0.00 8 47.69 52.31 9 10 11 12 13 14 15 16 Total

French dataset. Comparison of methods Cumulative JRC N=8 N=12 N=16 Current Raw data 4 58.13 64.03 67.63 100.00 8 94.74 96.75 12 - 99.78 16 95% centered data 79.52 82.75 85.79 98.83 99.42 99.99 96% centered data 64.26 70.18 73.02 94.32 95.88 99.45 97% centered data 43.85 49.74 52.27 99.85 99.94 79.03 82.71 99.88 99.96 92.99 99.97 Cumulative JRC N=8 N=12 N=16 Current 98% centered data 4 23.99 26.65 29.64 99.14 8 98.59 46.35 51.74 12 - 99.13 64.19 16 99.41 99% centered data 9.38 10.63 12.18 94.16 84.41 16.27 19.08 94.17 86.85 22.53 89.08 100% centered data 2.56 3.04 3.18 77.72 41.27 3.75 4.04 77.76 38.03 4.19 35.69 101% centered data 0.47 0.50 0.61 47.66 8.12 0.53 0.66 47.69 4.28 2.83

Assessment of JRC method

French dataset. Assessment of JRC method (A=1.01) Raw data N. tests Pass Rate (%) Fail Rate 3 58.43 0.00 4 22.96 Cumulative 81.38 5 10.78 6 4.75 7 1.96 8 0.73 99.60 9 0.27 10 0.09 11 0.02 12 100.00 13 14 15 16 Total Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 78.94 0.00 4 14.15 Cumulative 93.08 5 4.52 6 1.75 7 0.44 8 0.14 99.93 9 0.05 10 0.01 11 12 100.00 13 14 15 16 Total Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 68.02 0.00 4 18.16 Cumulative 86.18 5 7.41 6 3.31 7 1.73 8 0.89 99.52 9 0.30 10 0.07 11 12 0.02 99.98 13 0.01 14 15 16 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 52.22 0.00 4 20.52 Cumulative 72.74 5 10.28 6 6.40 7 4.06 8 2.40 95.88 9 1.53 10 1.08 11 0.61 12 0.37 99.47 13 0.24 14 0.12 15 0.05 16 0.10 Total 100.00

French dataset. Assessment of JRC method (A=1.01) Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 35.75 0.00 4 17.06 Cumulative 52.81 5 11.77 6 8.00 7 5.87 8 4.58 83.02 0.01 9 3.55 10 2.75 11 2.22 12 1.68 93.21 13 1.29 14 1.12 15 0.89 16 3.48 Total 99.99 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 20.06 0.27 4 9.78 0.13 Cumulative 29.84 0.4 5 7.14 0.11 6 5.97 0.06 7 5.03 0.03 8 4.24 0.02 52.22 0.61 9 3.61 10 3.21 0.00 11 2.85 0.01 12 2.68 64.57 0.65 13 2.31 14 2.17 15 1.93 16 28.36 Total 99.33 0.67 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 8.75 1.96 4 3.51 1.35 Cumulative 12.26 3.31 5 2.36 1.3 6 1.79 1.06 7 1.48 0.87 8 1.18 0.80 19.07 7.33 9 1.09 0.77 10 0.82 0.64 11 0.52 12 0.66 0.45 22.40 9.71 13 0.56 0.41 14 0.53 0.42 15 0.60 0.43 16 64.55 0.39 Total 88.64 11.36 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 2.38 8.79 4 0.72 7.20 Cumulative 3.10 15.99 5 0.45 6.8 6 0.26 5.82 7 0.10 5.42 8 0.12 4.63 4.03 38.67 9 0.03 4.26 10 0.02 3.82 11 3.53 12 0.01 3.17 4.12 53.45 13 3.02 14 0.00 2.69 15 2.44 16 32.01 2.25 Total 36.16 63.84

French dataset. Assessment of JRC method Cumulative A=1.00 A=1.01 Raw data 4 67.63 81.38 8 96.75 99.60 12 99.78 100.00 16 95% centered data 85.79 93.08 99.42 99.93 99.99 96% centered data 73.02 86.18 95.88 99.52 99.45 99.98 97% centered data 52.27 72.74 82.71 92.99 99.47 99.97 Cumulative A=1.00 A=1.01 98% centered data 4 29.64 52.81 8 51.74 83.02 12 64.19 93.21 16 99.41 99.99 99% centered data 12.18 29.84 19.08 52.22 22.53 64.57 89.08 99.33 100% centered data 3.18 12.26 4.04 19.07 4.19 22.40 35.69 88.64 101% centered data 0.61 3.10 0.66 4.03 4.12 2.83 36.16

Number of families which have 3,4,…,18 vehicles German dataset. Preprocessing We use 987 vehicles from the 186 families which have at least 3 vehicles Number of families which have 3,4,…,18 vehicles 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 95 2 Raw data Data shifted to 100% min 1st Qu median mean 3rd Qu max sd 85.47 92.58 94.86 95.01 97.27 105.26 3.44 min 1st Qu median mean 3rd Qu max sd 93.55 99.02 100.03 100.00 100.99 106.22 1.70

German dataset. Assessment of JRC method (16 vehicle max) Raw data N. tests Pass Rate (%) Fail Rate 3 38.97 0.04 4 16.55 0.00 Cumulative 55.52 5 10.96 6 8.23 7 6.29 8 4.82 85.82 0.05 9 3.47 10 2.68 11 2.12 12 1.56 95.65 13 1.16 14 0.87 15 0.60 16 1.67 Total 99.95 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 83.70 0.00 4 11.88 Cumulative 95.57 5 3.23 6 0.97 7 0.19 8 0.03 99.99 9 0.01 10 11 12 100.00 13 14 15 16 Total Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 71.14 0.00 4 17.02 Cumulative 88.16 5 6.61 6 3.00 7 1.30 8 0.57 99.65 9 0.22 10 0.09 11 0.02 12 0.01 100.00 13 14 15 16 Total Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 53.42 0.00 4 19.31 Cumulative 72.73 5 10.44 6 6.06 7 3.80 8 2.52 95.55 9 1.65 10 1.01 11 0.70 12 0.42 99.32 13 0.25 14 0.17 15 0.09 16 Total 100.00

German dataset. Assessment of JRC method (16 vehicle max) Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 31.34 0.06 4 15.22 0.02 Cumulative 46.56 0.08 5 10.08 0.01 6 7.70 0.00 7 5.87 8 4.73 74.93 0.09 9 3.83 10 3.20 11 2.65 12 2.27 86.89 13 1.93 14 1.60 15 1.39 16 8.10 Total 99.91 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 12.01 0.83 4 5.62 0.58 Cumulative 17.62 1.41 5 4.26 0.49 6 3.33 0.43 7 2.67 0.34 8 2.30 0.32 30.18 2.99 9 2.00 0.26 10 1.81 0.20 11 1.62 0.19 12 1.47 0.16 37.09 3.81 13 1.31 0.14 14 1.17 0.12 15 1.18 0.10 16 54.99 Total 95.73 4.27 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.46 8.08 4 0.87 7.17 Cumulative 3.33 15.24 5 0.48 6.54 6 0.24 5.70 7 0.17 5.10 8 0.11 4.76 4.34 37.34 9 0.09 4.10 10 0.06 3.72 11 0.04 3.59 12 0.03 3.19 4.56 51.94 13 0.02 2.9 14 2.79 15 2.66 16 32.70 2.4 Total 37.32 62.68 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.40 31.43 4 0.06 20.31 Cumulative 0.45 51.74 5 0.01 14.92 6 0.00 10.49 7 6.93 8 4.78 0.47 88.87 9 3.33 10 2.32 11 1.64 12 1.11 97.27 13 0.75 14 0.51 15 0.36 16 0.43 0.21 Total 0.90 99.10

German dataset. Assessment of JRC method (12 vehicle max) Raw data N. tests Pass Rate (%) Fail Rate 3 36.91 0.03 4 15.76 0.00 Cumulative 52.67 5 10.05 0.01 6 7.85 7 6.59 8 5.04 82.22 0.04 9 3.88 10 2.82 11 2.28 12 8.77 Total 99.96 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 82.06 0.00 4 12.75 Cumulative 94.82 5 3.58 6 1.14 7 0.37 8 0.08 99.98 9 0.01 10 11 12 Total 100.00 Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 69.11 0.00 4 17.42 Cumulative 86.54 5 6.85 6 3.36 7 1.77 8 0.80 99.32 9 0.40 10 0.15 11 0.07 12 0.06 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 50.88 0.00 4 18.72 Cumulative 69.60 5 10.99 6 6.29 7 4.13 8 2.80 93.80 9 2.09 10 1.40 11 0.86 12 1.85 Total 100.00

German dataset. Assessment of JRC method (12 vehicle max) Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 29.54 0.05 4 14.08 0.03 Cumulative 43.62 0.08 5 9.69 0.01 6 7.01 7 5.56 8 4.56 70.44 0.11 9 3.69 0.00 10 3.17 11 2.88 12 19.70 Total 99.88 0.12 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 11.18 0.86 4 4.95 0.76 Cumulative 16.13 1.62 5 3.66 0.72 6 2.65 0.59 7 2.20 0.51 8 1.83 0.5 26.47 3.94 9 1.58 0.49 10 1.33 0.34 11 1.19 0.33 12 64.01 Total 94.58 5.42 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.29 9.25 4 0.73 8.91 Cumulative 3.02 18.17 5 0.36 7.71 6 0.19 6.85 7 0.13 6.16 8 0.05 5.59 3.75 44.48 9 0.04 4.89 10 0.02 4.38 11 0.03 4.17 12 34.80 3.43 Total 38.65 61.35 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.34 34.46 4 0.07 21.54 Cumulative 0.41 56 5 0.02 15.29 6 0.01 9.99 7 0.00 6.67 8 4.28 0.43 92.22 9 2.63 10 1.78 11 1.14 12 1.08 0.71 Total 1.52 98.48

German dataset. Assessment of JRC method (8 vehicle max) Raw data N. tests Pass Rate (%) Fail Rate 3 33.46 0.06 4 13.90 0.01 Cumulative 47.36 0.07 5 9.37 6 6.92 0.00 7 5.72 8 30.54 Total 99.92 0.08 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 78.33 0.00 4 13.80 Cumulative 92.13 5 4.80 6 1.92 7 0.77 8 0.37 Total 100.00 Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 65.25 0.00 4 17.59 Cumulative 82.84 5 7.94 6 3.99 7 2.33 8 2.90 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 47.30 0.00 4 17.92 Cumulative 65.22 5 10.14 6 6.72 7 4.57 8 13.35 Total 100.00

German dataset. Assessment of JRC method (8 vehicle max) Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 26.79 0.09 4 11.92 0.05 Cumulative 38.71 0.15 5 8.18 6 6.04 0.04 7 4.78 0.02 8 42.03 0.01 Total 99.73 0.27 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 9.91 1.31 4 3.98 1.43 Cumulative 13.89 2.73 5 2.63 1.46 6 1.91 1.34 7 1.50 1.23 8 72.11 1.21 Total 92.04 7.96 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.08 11.56 4 0.49 12.17 Cumulative 2.57 23.73 5 0.26 10.88 6 0.16 9.24 7 0.07 7.58 8 38.97 6.54 Total 42.03 57.97 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.31 39.32 4 0.05 25.18 Cumulative 0.35 64.50 5 0.01 15.71 6 0.00 8.49 7 5.01 8 3.35 2.58 Total 3.71 96.29

Number of families which have 3,4,…,47 vehicles Dutch dataset. Preprocessing We use 471 vehicles from the 49 families which have at least 3 vehicles Number of families which have 3,4,…,47 vehicles 3 4 5 6 7 8 9 10 11 12 13 16 17 21 25 32 34 47 1 2 Raw data Data shifted to 100% min 1st Qu median mean 3rd Qu max sd 84.03 92.45 94.73 94.70 96.85 102.46 2.75 min 1st Qu median mean 3rd Qu max sd 90.88 98.88 99.98 100.00 101.34 105.40 1.85

Dutch dataset. Assessment of JRC method (16 vehicle max) Raw data N. tests Pass Rate (%) Fail Rate 3 50.72 0.00 4 23.23 Cumulative 73.95 5 13.59 6 6.46 7 3.17 8 1.52 98.69 9 0.73 10 0.33 11 0.14 12 0.06 99.94 13 0.04 14 0.02 15 16 Total 100.00 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 80.98 0.00 4 13.73 Cumulative 94.71 5 3.99 6 1.03 7 0.22 8 0.05 100.00 9 10 11 12 13 14 15 16 Total Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 65.36 0.00 4 21.36 Cumulative 86.72 5 7.19 6 3.39 7 1.67 8 0.65 99.62 9 0.22 10 0.11 11 0.03 12 0.01 99.99 13 14 15 16 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 44.21 0.00 4 22.09 Cumulative 66.30 5 12.85 6 7.54 7 4.52 8 2.88 94.10 9 2.08 10 1.35 11 0.95 12 0.60 99.07 13 0.36 14 0.24 15 0.13 16 0.21 Total 100.00

Dutch dataset. Assessment of JRC method (16 vehicle max) Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 25.49 0.1 4 11.48 0.04 Cumulative 36.97 0.15 5 9.81 0.02 6 8.04 7 6.54 8 5.28 66.63 0.17 9 4.61 10 3.79 11 3.29 12 2.85 81.18 13 2.46 14 2.12 15 1.84 16 12.23 Total 99.83 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 11.30 1.70 4 4.88 0.82 Cumulative 16.19 2.53 5 3.15 0.66 6 2.29 0.59 7 1.90 0.44 8 1.71 0.38 25.25 4.60 9 1.52 0.30 10 1.40 0.27 11 1.23 0.25 12 1.06 0.21 30.45 5.63 13 0.99 0.15 14 0.94 0.18 15 0.88 0.16 16 60.48 0.14 Total 93.75 6.25 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.55 8.19 4 0.89 7.23 Cumulative 3.44 15.43 5 0.47 6.62 6 0.28 5.86 7 0.18 5.35 8 0.12 4.76 4.50 38.01 9 0.08 4.21 10 0.05 3.91 11 0.03 3.45 12 3.29 4.68 52.87 13 0.01 2.98 14 2.72 15 2.59 16 31.74 2.39 Total 36.45 63.55 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.43 28.76 4 0.06 20.32 Cumulative 0.49 49.08 5 0.02 15.57 6 0.00 10.26 7 7.05 8 4.84 0.53 86.81 9 3.39 10 2.51 11 1.96 12 1.36 96.03 13 0.97 14 0.72 15 16 0.88 0.35 Total 1.41 98.59

Dutch dataset. Assessment of JRC method (12 vehicle max) Raw data N. tests Pass Rate (%) Fail Rate 3 47.96 0.00 4 22.74 Cumulative 70.70 5 13.43 6 7.57 7 4.01 8 2.04 97.75 9 1.12 10 0.60 11 0.27 12 0.26 Total 100.00 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 79.01 0.00 4 14.63 Cumulative 93.63 5 4.35 6 1.55 7 0.32 8 0.13 99.98 9 0.01 10 11 12 Total 100.00 Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 62.03 0.00 4 22.06 Cumulative 84.09 5 8.06 6 3.83 7 2.25 8 1.02 99.25 9 0.39 10 0.22 11 0.07 12 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 41.98 0.00 4 20.65 Cumulative 62.63 5 12.95 6 7.91 7 5.01 8 3.46 91.96 9 2.27 10 1.70 11 1.29 12 2.78 Total 100.00

Dutch dataset. Assessment of JRC method (12 vehicle max) Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 24.24 0.10 4 10.49 0.05 Cumulative 34.73 0.15 5 8.49 0.02 6 7.14 0.04 7 6.03 0.00 8 5.10 61.49 0.21 9 4.39 10 3.62 11 3.08 12 27.21 Total 99.78 0.22 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 10.17 1.92 4 4.35 0.99 Cumulative 14.52 2.91 5 2.71 0.96 6 2.16 0.83 7 1.56 0.62 8 1.24 0.56 22.18 5.88 9 1.10 0.63 10 1.01 0.46 11 0.90 0.42 12 67.02 0.39 Total 92.21 7.79 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.46 9.03 4 0.83 Cumulative 3.29 18.06 5 0.40 8.05 6 0.20 7.05 7 0.13 6.41 8 0.06 5.48 4.06 45.05 9 4.77 10 0.02 4.47 11 0.03 4.04 12 33.82 3.68 Total 37.99 62.01 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.43 30.61 4 0.05 22.18 Cumulative 0.48 52.79 5 0.02 16.23 6 0.00 9.89 7 6.62 8 4.49 0.49 90.02 9 2.97 10 2.01 11 1.45 12 1.04 Total 2.51 97.49

Dutch dataset. Assessment of JRC method (8 vehicle max) Raw data N. tests Pass Rate (%) Fail Rate 3 43.65 0.00 4 20.57 Cumulative 64.22 5 13.26 6 8.52 7 5.30 8 8.71 Total 100.00 Data centered around 95% N. tests Pass Rate (%) Fail Rate 3 75.16 0.00 4 15.81 Cumulative 90.97 5 5.11 6 2.60 7 0.83 8 0.49 Total 100.00 Data centered around 96% N. tests Pass Rate (%) Fail Rate 3 57.10 0.00 4 21.81 Cumulative 78.91 5 9.84 6 4.55 7 2.80 8 3.89 Total 100.00 Data centered around 97% N. tests Pass Rate (%) Fail Rate 3 38.46 0.00 4 17.67 Cumulative 56.12 5 12.36 6 8.18 7 5.80 8 17.53 Total 100.00

Dutch dataset. Assessment of JRC method (8 vehicle max) Data centered around 98% N. tests Pass Rate (%) Fail Rate 3 21.66 0.17 4 8.22 0.07 Cumulative 29.89 0.24 5 6.79 0.05 6 5.86 0.03 7 4.44 0.02 8 52.67 0.01 Total 99.65 0.35 Data centered around 99% N. tests Pass Rate (%) Fail Rate 3 9.44 2.31 4 3.63 1.58 Cumulative 13.07 3.88 5 2.08 1.68 6 1.32 1.51 7 1.04 1.42 8 72.62 1.39 Total 90.12 9.88 Data centered around 100% N. tests Pass Rate (%) Fail Rate 3 2.28 11.07 4 0.56 12.01 Cumulative 2.84 23.08 5 0.23 6 0.11 9.42 7 0.08 7.68 8 38.78 6.70 Total 42.05 57.95 Data centered around 101% N. tests Pass Rate (%) Fail Rate 3 0.38 36.62 4 0.05 26.22 Cumulative 0.43 62.84 5 0.01 15.54 6 0.00 8.81 7 5.01 8 4.27 3.08 Total 4.71 95.29

Conclusions The presentation provides a more extensive analysis of the CoP data submitted by France, Germany and The Netherlands in particular for what concerns the assessment of the JRC proposal for the CoP of CO2/FC. Main results in addition to previous analysis: JRC approach appears to confirm its robustness and the higher flexibility compared to the current CO2 CoP procedure (trade-off between margin of safety and number of tests) As the mean of the distribution gets closer to the pass limit, the JRC approach allows a slightly better protection from false fails The JRC also provides a better protection from false passes On the bases of the CoP test data that have been submitted and analysed, changing the max number of tests to 12 and 8 does not significantly affect the results, suggesting that also a lower than 16 max number of tests could be taken for the final pass/fail decision

Thanks Questions? You can find me at Biagio.CIUFFO@ec.europa.eu