Digital Camera Field Testing Results in Support of the Digital Opacity Compliance System (DOCS) Michael J. McFarland (Utah State University) Steve L. Rasmussen (Hill AFB) Daniel A. Stone (Hill AFB) Glenn R. Palmer (Hill AFB) Mike Spencer (Eastman Kodak, Inc.) Josh A. Gunter (EMassist, Inc.) Joint Services Environmental Management Conference & Exposition April 13, 2005

OUTLINE Background Theory of DOCS Goal and Objectives Methods Hardware Validation Field Results Conclusions and Recommendations

Acknowledgements Field demonstration was financially supported by the Environmental Security Technology Certification Program (Contract CP-200119)

Acknowledgements Participating Organizations Included: EPA Emission Measurement Center EPA Regions VI, VIII California Air Resources Board Texas Commission on Environmental Quality Utah Division of Air Quality Hill AFB, UT Eastman Kodak, Inc. (Rochester, NY)

BACKGROUND Visible opacity is the most frequently cited air quality parameter in Title V operating permits. For the majority of regulated air sources, the primary method for determining compliance with permitted opacity levels is EPA Reference Method 9 (Method 9)

BACKGROUND Method 9 relies on trained human observers to visually determine compliance by estimating the opacity of a smoke plume once every 15 seconds for a specified time period. Because its opacity estimates are inherently subjective, Method 9 results are vulnerable to claims of inaccuracy, bias and, in some cases, outright fraud.

BACKGROUND Beyond the technical concerns, the recurring Method 9 training costs can become a significant financial burden on a facility’s compliance budget. The Department of Defense (DoD) spends approximately $42 million dollars per year to support the Method 9 compliance program.

BACKGROUND With declining DoD compliance budgets, development of cost-effective and regulatory supportable methods for verifying visible opacity is receiving increased attention.

Digital Opacity Compliance System The Digital Opacity Compliance System (DOCS) is a low cost and innovative technology that employs digital imaging technology for quantifying visible opacity Recent economic analyses have demonstrated that adoption of the DOCS technology could save DoD approximately $16 million dollars per year in air compliance costs.

Digital Opacity Compliance System The DOCS uses a commercial-off-the-shelf (COTS) digital camera to capture images of visible opacity, which are then analyzed using a simple computer software package

Basics of the DOCS Technology

Digital Opacity Compliance System Operational Steps Activate the DOCS opacity analysis program Retrieve those digital photographs to be evaluated Draw an analysis box (or grid) around that portion of the visible emissions that will be analyzed for opacity

Previous Field Demonstrations From Jan. 2001 through Dec. 2003, the DOCS technology was field tested at three EPA-approved smoke schools (Ogden, UT, Augusta, GA and Columbus, OH) as well as a number of military and private commercial sites. Earlier field data confirmed that the DOCS consistently met the performance standards established for Method 9

Field Demonstration of the DOCS at Sludge Incinerator in Healy, Alaska

Validated DOCS Hardware The validated digital cameras used to support the DOCS field demonstrations included the Kodak models DC265 and DC290. Both digital camera models are no longer commercially available. Without validated hardware, DOCS technology users can not be supported

Goal of Study The goal of the current effort was to develop, implement and evaluate a series of commercially available digital cameras in support of the DOCS technology

OBJECTIVES Evaluate commercially available digital cameras for their ability to support the DOCS technology software. Statistically compare the opacity results obtained from digital cameras to those reported by EPA-certified in-stack transmissometers and previously validated digital camera systems.

METHODS Based on price and functionality, the Kodak model DX6490, Sony Cybershot DSC-WI, Fuji Finepix E500 and the Nikon Coolpix 5200 were selected to undergo field validation testing. Kodak Model DX6490 was evaluated against the Kodak Model DC290 at an EPA-approved Method 9 smoke school held in Syracuse, NY

METHODS The digital photographs of visible opacity taken by the Kodak model DX6490 and DC290 were evaluated using the DOCS technology. The field results were compared to the visible opacity reported by the EPA-certified in-stack transmissometer

Opacity (Transmissometer) – Opacity (DOCS) METHODS The average opacity difference for both the Kodak model DX6490 and DC290 was computed using the following equation: Opacity (Transmissometer) – Opacity (DOCS)

METHODS The field data was analyzed statistically using a paired t-test and by computing the 99% confidence interval about the average opacity difference.

METHODS The Sony Cybershot DSC-WI, Fuji Finepix E500 and the Nikon Coolpix 5200 were evaluated using a Method 9 smoke generator in Anchorage, Alaska.

METHODS Since an EPA-certified transmissometer was not available, the opacity results collected from the new digital cameras were compared to results obtained from the Kodak model DC290.

Opacity (DC290) – Opacity (new systems) METHODS The average opacity difference for between the Kodak model DC290 and the new digital camera models was computed using the following equation: Opacity (DC290) – Opacity (new systems)

METHODS As in the Syracuse, NY Method 9 smoke school, the field data was analyzed statistically by computing the 99% confidence interval about the average opacity difference.

Positioning of DOCS Cameras STACK Stack Height – 15 Feet Stack Distance – 50 feet C1 70 degrees 70 degrees C4 C3 C2 70 degrees 70 degrees

(Transmissometer – Camera) RESULTS Performance of the Kodak DC290 and Kodak DX6490 at Method 9 Smoke School (0 – 100%) Camera Model Opacity Range Plume Photos (n) Ave. Difference % (Transmissometer – Camera) 99% CI Significant? Kodak DC290 0-100% 100 -0.72 -4.24<-0.72<2.79 No Kodak DX6490 -7.30 -12.74<-7.30<-1.85 Yes

Results suggest that from 0 – 100% opacity, the Kodak model DC290 was statistically equivalent to the EPA-certified transmissometer. The Kodak model DX6450 was not equivalent.

(Transmissometer – Camera) RESULTS Performance of the Kodak DC290 and Kodak DX6490 at Method 9 Smoke School (0 - 40%) Camera Model Opacity Range Plume Photos (n) Ave. Difference % (Transmissometer – Camera) 99% CI Significant? Kodak DC290 0-40% 55 -1.36 -4.91<-1.36<2.19 No Kodak DX6490 -7.87 -17.77<-7.87<2.04

Results suggest that over the opacity range of 0 – 40%, both the Kodak model DC290 and DX6450 were statistically equivalent to the EPA certified transmissometer.

Performance of the Kodak DC290 and New Digital Camera Models (0-100%) RESULTS Performance of the Kodak DC290 and New Digital Camera Models (0-100%) Camera Model Opacity Range No. of Photos (n) Ave. Difference % (Kodak DC290 – New Camera) 99% CI Significant? Sony Cybershot DSC-WI 0-100% 810 2.42 1.16<2.42<3.68 Yes Fuji Finepix E500 -1.13 -2.2<-1.13<-0.05 Nikon Coolpix 5200 1.85 0.65<1.85<3.05

Results suggest that over the full range of opacity (0 – 100%), none of the new camera models were statistically equivalent to the Kodak model DC290.

Performance of the Kodak DC290 and New Digital Camera Models (0 – 40%) RESULTS Performance of the Kodak DC290 and New Digital Camera Models (0 – 40%) Camera Model Opacity Range No. of Photos (n) Ave. Difference % (Kodak DC290 – New Camera) 99% CI Significant? Sony Cybershot DSC-WI 0-40% 679 0.11 -0.81<0.11<1.04 No Fuji Finepix E500 -2.20 -2.98<-2.20<-1.42 Yes Nikon Coolpix 5200 -0.46 -1.24<-0.46<0.32

Results suggest that over the opacity range of 0 – 40%, the Sony Cybershot DSC-WI and the Nikon Coolpix 5200 were statistically equivalent to the Kodak model DC290.

Over the 0 – 40% opacity range, the performance of the Fuji Finepix E500 was significantly different.

CONCLUSIONS Over the entire opacity range, the Kodak DC290-DOCS combination was found to measure visible opacity with an accuracy statistically equivalent to the EPA-certified transmissometer.

CONCLUSIONS Over the 0 – 40% opacity range, the performance of the Kodak model DX6490, Sony Cybershot DSC-WI and Nikon Coolpix 5200 was found to be comparable to the Kodak DC290

CONCLUSIONS The Fuji Finepix E500-DOCS combination measured visible opacity with an accuracy that was statistically different from the Kodak model DC290.

RECOMMENDATIONS Because of rapid improvements in digital technology, any regulatory approved opacity method that employs digital imagery must incorporate a protocol for testing and validating new hardware. Camera validation testing should be expanded to include a greater number of digital camera makes/models

RECOMMENDATIONS For the DoD (and others) in the regulated community to recognize the financial benefit of adopting digital technology for opacity compliance verification, development and promulgation of a new regulatory approved method is urgently needed.

QUESTIONS?