1. OBD Program Design and Performance Mobile Sources Program Air Pollution Control Division

2.  In 2011 approaching expiration of the I/M contract made a natural point to consider program changes.  Staff reviewed available I/M program effectiveness studies.  Staff initiated a study to explore effectiveness and costs of program design factors. 2

3.  National Academies of Science Report: “Evaluating Emissions Inspection and Maintenance Programs” (2001)  On-Board Diagnostics Policy Work Group report (2002)  EPA: “High Mileage Study of On-Board Diagnostics Emissions” (2005)  Gardetto and Lawson: response to “High Mileage Study of On-Board Diagnostics Emissions” (2006)  CARB: “Transitioning Away from Smog Check Tailpipe Emissions Testing in California for OBDII Equipped Vehicles” (2009)  Georgia Tech Research Institute:“Estimated Validity and Reliability of On- Board Diagnostics for Older Vehicles: Comparison with Remote Sensing Observations” (2011) 3/08/2012 3 Vehicle IM Program Review and Evaluation Part 2

4.  Existing I/M Programs reducing emissions less than MOBILE model predicts ◦ Lowest benefit for test and repair idle tests ◦ Highest benefit for centralized transient tests ◦ Modeled benefit subsequently substantially reduced in MOBILE6 model version  I/M Programs should be evaluated using actual test data (program test data, RSD data, pull-over studies)  Data for evaluating OBDII is inadequate ◦ Concern with lack of overlap between IM 240 and OBDII failures ◦ Concern over large percentage of OBDII failures with low emissions 3/08/2012 4 Vehicle IM Program Review and Evaluation Part 2

5.  There continues to be limited publicly available data examining the comparative effectiveness of OBDII and IM240 Programs.  MOVES model provides greatest credit for OBDII program with IM240 testing for model year 1995 and older vehicles  Colorado lab study and program data has shown that IM240 is an effective way to identify high emitting vehicles but will miss some high emitting vehicles identified by OBDII.  Studies and Colorado program data show that OBDII effectively identifies high emitting vehicles, but will miss some high emitting identified by IM 240. 3/08/2012 Vehicle IM Program Review and Evaluation Part 25

6. Colorado Air Quality Challenge

7. 3 Year Average of 4 th Max. 8 Hour Ozone 0.080 ppm std. Site Name2002- 2004 2003- 2005 2004- 2006 2005- 2007 2006- 2008 2007- 2009 2008 - 2010 2009 - 2011 2010 - 2012 2011 - 2013 2012- 2014 2013- 2015 Welby0.0660.0680.0690.0700.0710.0720.070 0.0710.0760.0730.071 Highland0.0790.0810.0770.078---0.0740.0770.079-- S. Boulder Creek0.0760.075 0.081 0.0780.073 0.0740.0770.0750.074 Carriage0.0740.0750.0700.0740.0730.0700.0680.0690.073--- Chatfield State Park*--0.0810.0840.0820.0770.0760.0770.0820.0830.0800.079 USAF0.0730.0740.073 0.0710.0670.0660.0670.0720.0740.0710.068 Manitou Springs--0.0720.0740.0730.069 0.0700.074 0.0690.066 Arvada0.0730.075 0.0790.0780.0740.0730.074---- Welch0.0690.0670.0690.0750.0780.0740.0710.0730.0760.0780.0750.073 Rocky Flats0.0840.080 0.0850.0860.0820.078 0.0800.0830.0820.079 NREL0.0830.0820.0780.0820.0810.0760.0720.0750.0790.0820.080 Rocky Mountain National Park 0.0820.0780.0740.076 0.0740.0730.0740.0770.0760.0730.070 Ft. Collins West----0.0820.0780.0740.0760.0780.0800.0790.077 Ft. Collins0.0700.0710.0720.0740.0710.0660.065 0.0690.0720.0730.071 Greeley – Weld Tower0.0770.076 0.0780.0760.071 0.0720.076 0.0740.072 *Chatfield Site Relocated in 2004

8. 3 Year Average of 4 th Max. 8 Hour Ozone 0.075 ppm std. Site Name2002- 2004 2003- 2005 2004- 2006 2005- 2007 2006- 2008 2007- 2009 2008 - 2010 2009 - 2011 2010 - 2012 2011 - 2013 2012- 2014 2013- 2015 Welby0.0660.0680.0690.0700.0710.0720.070 0.0710.0760.0730.071 Highland0.0790.0810.0770.078---0.0740.0770.079-- S. Boulder Creek0.0760.075 0.081 0.0780.073 0.0740.0770.0750.074 Carriage0.0740.0750.0700.0740.0730.0700.0680.0690.073--- Chatfield State Park*--0.0810.0840.0820.0770.0760.0770.0820.0830.0800.079 USAF0.0730.0740.073 0.0710.0670.0660.0670.0720.0740.0710.068 Manitou Springs--0.0720.0740.0730.069 0.0700.074 0.0690.066 Arvada0.0730.075 0.0790.0780.0740.0730.074---- Welch0.0690.0670.0690.0750.0780.0740.0710.0730.0760.0780.0750.073 Rocky Flats0.0840.080 0.0850.0860.0820.078 0.0800.0830.0820.079 NREL0.0830.0820.0780.0820.0810.0760.0720.0750.0790.0820.080 Rocky Mountain National Park 0.0820.0780.0740.076 0.0740.0730.0740.0770.0760.0730.070 Ft. Collins West----0.0820.0780.0740.0760.0780.0800.0790.077 Ft. Collins0.0700.0710.0720.0740.0710.0660.065 0.0690.0720.0730.071 Greeley – Weld Tower0.0770.076 0.0780.0760.071 0.0720.076 0.0740.072 *Chatfield Site Relocated in 2004

9. 3 Year Average of 4 th Max. 8 Hour Ozone 0.070 ppm std. Site Name2002- 2004 2003- 2005 2004- 2006 2005- 2007 2006- 2008 2007- 2009 2008 - 2010 2009 - 2011 2010 - 2012 2011 - 2013 2012- 2014 2013- 2015 Welby0.0660.0680.0690.0700.0710.0720.070 0.0710.0760.0730.071 Highland0.0790.0810.0770.078---0.0740.0770.079-- S. Boulder Creek0.0760.075 0.081 0.0780.073 0.0740.0770.0750.074 Carriage0.0740.0750.0700.0740.0730.0700.0680.0690.073--- Chatfield State Park*--0.0810.0840.0820.0770.0760.0770.0820.0830.0800.079 USAF0.0730.0740.073 0.0710.0670.0660.0670.0720.0740.0710.068 Manitou Springs--0.0720.0740.0730.069 0.0700.074 0.0690.066 Arvada0.0730.075 0.0790.0780.0740.0730.074---- Welch0.0690.0670.0690.0750.0780.0740.0710.0730.0760.0780.0750.073 Rocky Flats0.0840.080 0.0850.0860.0820.078 0.0800.0830.0820.079 NREL0.0830.0820.0780.0820.0810.0760.0720.0750.0790.0820.080 Rocky Mountain National Park 0.0820.0780.0740.076 0.0740.0730.0740.0770.0760.0730.070 Ft. Collins West----0.0820.0780.0740.0760.0780.0800.0790.077 Ft. Collins0.0700.0710.0720.0740.0710.0660.065 0.0690.0720.0730.071 Greeley – Weld Tower0.0770.076 0.0780.0760.071 0.0720.076 0.0740.072 *Chatfield Site Relocated in 2004

10. FailPass Fail Stationary Idle Test ( BAR 90 ) Federal Test Procedure 0.11.01.210.0 1 10 100 500 7 CO measurements

11. Program Effectiveness

12.  Lane Data ◦ Mass based emissions (IM240). ◦ All failures given two back-to-back tests. (8 minutes total) ◦ All capable vehicles given OBD interrogation.  Remote Sensing Network ◦ ≈15-19 units/day. ◦ ≈5,000,000 valid readings/year. ◦ ≈265,000+ unique vehicles. ◦ 2% of clean-screened vehicles held back – have to go to “brick and mortar.”  2% Study ◦ Random sample of Tier II vehicles given two full back-to-back IM240 tests (no fast-pass) + OBD interrogation. Vehicles stay on this track. 12

13.  Laboratory studies ◦ Recruit vehicles from lanes or through mailers  Compare/crosscheck this data in various ways - lab results to RSD, IM240 and OBD, RSD to IM240, IM240 to OBD, etc. 13

14. 14

15.  Will only focus on possible errors of omission.  2014 Test Year Data ◦ Latest Colorado data available without pass/fail OBD. ◦ Assume the least incentive for motorist to interfere with OBD results. 15 % Fail for High Emissions Yet OBD MIL Command = Off42% Number of Vehicles Total Vehicles Tested717,195 Total Failing OBD73,247 Total Failing 24017,512 Total Passing Command, Passing MIL, Failing Emissions7,334

16.  Add Monitor Results ◦ Unset readiness indicates system testing not complete. ◦ Might indicate motorist interference.  Several options: ◦ “National” model (MY 2000 1 unset) Or ◦ EPA revised guidance (Same, but O2, O2 heater, catalyst must be ready) Or ◦ No unset readiness allowed 16 Number of Vehicles high Exhaust w/no MIL % Missed by OBD National Program5,91434% EPA Guidance Update4,35225% No Unset Readiness383222%

17.  Distribution by Model Year. ◦ MIL = Off, readiness not considered. (7334 vehicles) ◦ Separation is kinda, sorta the break between Tier I and Tier II 17 V_VEH_YEARCountOfV_VIN 1996641 1997842 19981112 19991103 2000991 2001505 2002543 2003298603582% 2004324 2005207 2006242 2007145 2008201 200970 201077 201121 201210 20132129918%

18.  What do the emissions look like? ◦ OBD pass with high emissions ◦ Reminder: IM240 values after two full B2B tests. Eight minutes total test time. 18 AvgOfV_CO g/m AvgOfV_NOX g/m AvgOfV_HC g/m National Program14.932.300.93 EPA Guidance Update14.122.260.87 No Unset Readiness13.892.320.85 CO g/m NOx g/m HC g/m 151.5 -2.51.2 Comparative IM240 Fail Cut-Points Certification Standards (it’s complicated) CO g/m NOx g/m NMOG g/m 4.20.070.09 CONOxNMHC g/m 4.20.60.31 Tier I Tier II

19. 19

20.  Recruit vehicles recently failed (within days) for: ◦ IM240 (exhaust only); ◦ OBD; ◦ Both.  After owner agrees to participate: ◦ Pre-inspection (no tampering accepted); ◦ Lane-grade IM240 & OBD test series; ◦ FTP test series; ◦ Repair; ◦ Repeat IM240, OBD, FTP series.

21. VEHID YEARCount 19974 19981 19997 20001 20013 20043 20051 20083 AvgOfFTP COAvgOfFTP NOxAvgOfFTP THC 8.361.730.58 11.050.981.55 8.592.680.87 18.991.110.48 6.560.990.81 28.270.961.24 12.342.171.11 17.310.510.45 21

22. Control #Model YearPart(s)Conclusion ECS-0021999CatalystNOx failure - not monitored this MY ECS-0141999Catalyst, O2, Plugs? ECS-0351997O2 Sensors? ECS-06520012 O2 Sensors MIL intermittant sometimes flashes, A/M cat, Cat monitor won't set ECS-0732000O2 Sensors? ECS-0942008 O2 sensors, catalystPattern failure ECS-1102004TSB ground repair? ECS-1422005CatalystsOne cat hollow 22