Quality Assurance in Thermography P. Plassmann C.D. Jones E.F.J. Ring University of Glamorgan, Faculty of Advanced Technology, UK R. Simpson National Physical Laboratory, Radiation Thermometry Group, UK

Introduction The rise, fall & rise of Thermography Another fall ahead? Quality Assurance of TI Introduction Introduction Protocol IR Cameras Tests Conclusions The rise, fall & rise of Thermography Another fall ahead? Reasons? Stability Range Uniformity Scene Spatial Res. Thermal Res. “Thermography” MedLine/Ovid citations

Introduction Past Reasons for Decline Future Threats ! Quality Assurance of TI Introduction Past Reasons for Decline Future Threats ! Thermography cover in US (Medicare) removed in 1990s Poor sensor quality Poor image processing techniques Inadequate physiological knowledge Expectations too high Indiscrete use and improper interpretation with no standardised protocols Introduction Protocol IR Cameras Tests Conclusions Stability Range Uniformity Scene Spatial Res. Thermal Res. Indiscrete use and improper interpretation with no standardised protocols

Introduction No Standardisation Means Quality Assurance of TI Introduction Introduction Protocol IR Cameras Tests Conclusions No Standardisation Means No reliable, quantifiable measurements Interpretation of results difficult Multi-centre trials unreliable Longitudinal trials difficult Acceptance of technique limited Stability Range Uniformity Scene Spatial Res. Thermal Res. Public image of medical thermographer

Protocol To Measure = To Compare Quality Assurance of TI Protocol To Measure = To Compare Standardisation Quality control Introduction Protocol IR Cameras Tests Conclusions Stability Range Uniformity Scene Spatial Res. Thermal Res. Therefore standardisation and quality control of IR Cameras Patient Imaging Analysis Storage File Exchange Presentation Protocols

Imagers Variety of Systems…. ….Common Problem: Quality Assurance of TI Imagers Introduction Protocol IR Cameras Tests Conclusions Variety of Systems…. ….Common Problem: Not designed for exact quantification in the medical range of temperatures Stability Range Uniformity Scene Spatial Res. Thermal Res.

4°C is 20% of human skin temperature range Quality Assurance of TI Manufacturer’s Specifications Introduction Protocol IR Cameras Tests Conclusions Accuracy (bias, offset) Most imagers have ± 2°C offset (Do you know how accurate yours is?) This is after Calibration (When was yours calibrated last?) Stability Range Uniformity Scene Spatial Res. Thermal Res. 4°C is 20% of human skin temperature range Resolution Spatial (128x128 or better) meaning? Thermal (most better 0.1°C) meaning?

Tests Stable Reference: Ice & Water Emissivity close to that of skin Quality Assurance of TI Tests Introduction Protocol IR Cameras Tests Conclusions Stable Reference: Ice & Water Emissivity close to that of skin Ice + water (almost) exactly 0º C Stability Range Uniformity Scene Spatial Res. Thermal Res. grey: ice, black: water

Stability Offset Drift Quality Assurance of TI Stability Introduction Protocol IR Cameras Tests Conclusions Offset Drift Cameras tested against UK’s National Physical Laboratory calibration source Stability Range Uniformity Scene Spatial Res. Thermal Res.

Stability Offset ‘wobble’ due to Internal Calibration Shutter Quality Assurance of TI Stability Offset ‘wobble’ due to Internal Calibration Shutter FLIR SC500 (before repair) Introduction Protocol IR Cameras Tests Conclusions Stability Range Uniformity Scene Spatial Res. Thermal Res. How to Test?

Stability TEST Preparation Room T stable (usual examination setting) Quality Assurance of TI Stability TEST Introduction Protocol IR Cameras Tests Conclusions Preparation Room T stable (usual examination setting) Set camera emissivity to 0.98 (for 3-5µm cameras) or 0.99 (for 9-12µm cameras) Distance ~ 1m Narrow T range around zero ºC (e.g. -5 to +5) Switch camera off, allow it to acclimatise to room temperature for at least one hour. Stability Range Uniformity Scene Spatial Res. Thermal Res.

Long-Term & Short-Term Stability Tests similar Quality Assurance of TI Stability TEST Introduction Protocol IR Cameras Tests Conclusions Method Fill bucket with ice/water mix, insulate Camera perpendicular to ice/water Switch camera on Stir bucket, baseline image of the ice/water Take images every 5 minutes (during the first 30 minutes, every 10 minutes after that) for a total of 2 hours. (Stir bucket before each image) Plot results Stability Range Uniformity Scene Spatial Res. Thermal Res. Long-Term & Short-Term Stability Tests similar

Range Offset neither Constant nor Linear ‘Human’ temperature range Quality Assurance of TI Range Introduction Protocol IR Cameras Tests Conclusions Offset neither Constant nor Linear ‘Human’ temperature range Stability Range Uniformity Scene Spatial Res. Thermal Res. Do you know yours?

Range TEST Preparation Room T stable (usual examination setting) Quality Assurance of TI Range TEST Introduction Protocol IR Cameras Tests Conclusions Preparation Room T stable (usual examination setting) Set camera emissivity to 0.98 (for 3-5µm cameras) or 0.99 (for 9-12µm cameras) Distance ~ 1m Switch camera on, allow it to settle (use data from ‘Stability’ experiment). T range 20ºC to 40ºC Container of water at about 40º C Platinum resistance or other calibrated thermometer required (ideally traceable to ITS90) Stability Range Uniformity Scene Spatial Res. Thermal Res.

Range TEST Method Stir it for at least 1 minute (use thermometer) Quality Assurance of TI Range TEST Introduction Protocol IR Cameras Tests Conclusions Method Stir it for at least 1 minute (use thermometer) Take 1st image + thermometer readout While the water is cooling, take images and thermometer readings every 0.5 ºC or so. Always stir the water for 1 minute before taking a reading/image. Add small amounts of cold water to speed-up the cooling. Plot result Stability Range Uniformity Scene Spatial Res. Thermal Res.

Uniformity Offset Variations due to Optical Limits Quality Assurance of TI Uniformity Introduction Protocol IR Cameras Tests Conclusions Offset Variations due to Optical Limits Corners most problematic Stability Range Uniformity Scene Spatial Res. Thermal Res. T=1.2°C Ever tested yours?

Uniformity TEST Preparation Room T stable (usual examination setting) Quality Assurance of TI Uniformity TEST Introduction Protocol IR Cameras Tests Conclusions Preparation Room T stable (usual examination setting) Set camera emissivity to 0.98 (for 3-5µm cameras) or 0.99 (for 9-12µm cameras) Distance ~ 1m, but de-focus max. Switch camera on, allow it to settle (use data from ‘Stability’ experiment). Wide diameter bucket or tray of water (water surface to entirely fill field of view) T ~ 23ºC (ca. mid range skin temperature) Narrow T range (21 to 25º C) Stability Range Uniformity Scene Spatial Res. Thermal Res.

Uniformity TEST Method Stir it for 30 seconds Quality Assurance of TI Uniformity TEST Introduction Protocol IR Cameras Tests Conclusions Method Stir it for 30 seconds Take an image (water surface only) Analyse image Stability Range Uniformity Scene Spatial Res. Thermal Res.

How does your Imager behave? Quality Assurance of TI Scene Introduction Protocol IR Cameras Tests Conclusions Offset Variations due to ‘Flooding’ Un-cooled imagers most at risk 30.19°C Stability Range Uniformity Scene Spatial Res. Thermal Res. 30.32°C 35.29°C T=0.13°C How does your Imager behave?

Scene TEST Preparation Room T stable (usual examination setting) Quality Assurance of TI Scene TEST Introduction Protocol IR Cameras Tests Conclusions Preparation Room T stable (usual examination setting) Set camera emissivity to 0.98 (for 3-5µm cameras) or 0.99 (for 9-12µm cameras) Distance ~ 1m Switch camera on, allow it to settle (use data from ‘Stability’ experiment). Bucket of water at room T Bucket of water at ~ 40ºC Stability Range Uniformity Scene Spatial Res. Thermal Res.

Scene TEST Method Take image 1 (room T water surface only) Quality Assurance of TI Scene TEST Introduction Protocol IR Cameras Tests Conclusions Method Take image 1 (room T water surface only) Take image 2 (room T water + part of 40º C surface) Analyse Stability Range Uniformity Scene Spatial Res. Thermal Res. 30.32°C 30.19°C

Spatial Resolution Temperature Peak Readout Error Quality Assurance of TI Spatial Resolution Introduction Protocol IR Cameras Tests Conclusions Temperature Peak Readout Error Establish minimum object size for correct temperature readout Similar to MFOV ‘split response’ method (Measurement Field of View) Stability Range Uniformity Scene Spatial Res. Thermal Res.

Spatial Resol. TEST Preparation Quality Assurance of TI Spatial Resol. TEST Preparation Room T stable, check emissivity, camera settled (as before) Distance ~ 30cm Bucket of water at ~ 40ºC Cardboard with  cutout (10cm x 2 cm, aluminium markers) Introduction Protocol IR Cameras Tests Conclusions Stability Range Uniformity Scene Spatial Res. Thermal Res.

Spatial Resol. TEST Method Quality Assurance of TI Spatial Resol. TEST Method Take image (note distance camera - cardboard, d1) Introduction Protocol IR Cameras Tests Conclusions produce cross-sections 3.1 cm z z = = 6.2mm 2cm x 3.1cm 10cm calc. width of “just about” cross-section, z Stability Range Uniformity Scene Spatial Res. Thermal Res. calculate width wmin for other distances d2 wmin= z x d2 d1 10 cm 2 cm

Thermal Resolution Noise + Digitisation Step Error Quality Assurance of TI Thermal Resolution Introduction Protocol IR Cameras Tests Conclusions Noise + Digitisation Step Error Usual measure: NETD (Noise Equivalent Temperature Difference), impractical: varies over time requires specialised equipment not standardised (ΔT 5°C or 10°C, object temp…) Instead: ThRes = digit.Step “+” noise Stability Range Uniformity Scene Spatial Res. Thermal Res. Digitisation Step Example: Measurement range: 25°C, Digitisation: 8 bit (256 steps) 1 digit step = 25°C / 256 0.1°C pixels 1 2 3 4 5 28.1°C 28.2°C 28.3°C cross-section

Std. Deviation TEST Preparation Method bucket of water, T ~ 30ºC Quality Assurance of TI Std. Deviation TEST Introduction Protocol IR Cameras Tests Conclusions Preparation bucket of water, T ~ 30ºC rest the same as under ‘Uniformity’ Stability Range Uniformity Scene Spatial Res. Thermal Res. Method Take image (camera de-focused max.) Calculate Standard Dev. min. AOI 10 x 10 pixels

Std. Deviation TEST Method (cont) Quality Assurance of TI Std. Deviation TEST Method (cont) Calculate Thermal Resolution (95% confidence) ThRespix = digStep² + (2 x std.Dev)² Introduction Protocol IR Cameras Tests Conclusions Stability Range Uniformity Scene Spatial Res. Thermal Res. real T2 std.dev. digStep real T1 Example (95% confidence): ThRespix = 0.095² +4 x 0.14² = 0.3 °C

Thermal Resolution Consequences Don’t use spot measurements ! Quality Assurance of TI Thermal Resolution Consequences Don’t use spot measurements ! Always use AOIs error halves with double amount of pixels squared therefore: 1 pixel: ± 0.3°C 4 pixels: ± 0.15°C 16 pixels: ± 0.075°C 64 pixels: ± 0.0375°C 256 pixels: ± 0.01875°C 8 bits per pixel are sufficient manufacturer quoted resolution not that important Introduction Protocol IR Cameras Tests Conclusions Stability Range Uniformity Scene Spatial Res. Thermal Res.

Conclusion (1 of 2) Current IR Imagers are Deficient Specifications Quality Assurance of TI Conclusion (1 of 2) Current IR Imagers are Deficient Specifications - offset of ±2°C even after manufacturer calibration Stability - short-term drift after internal calibration possible (±0.3°C) - medium term, even after switch-on time specified by manufacturer, (±1°C) - long-term, over several hours/days (±0.5°C) Range - fluctuation of around ±1°C possible within ‘human’ range Uniformity - optical limitations typically ±0.5°C Introduction Protocol IR Cameras Tests Conclusions Stability Range Uniformity Scene Spatial Res. Thermal Res.

Conclusion (2 of 2) Current IR Imagers are Deficient Scene Quality Assurance of TI Conclusion (2 of 2) Current IR Imagers are Deficient Scene - flooding effect ±0.2°C Spatial Resolution - border / slit effect (not quantifiable, just don’t do it) Thermal Resolution - digitisation ‘+’ noise effect ±0.3°C Introduction Protocol IR Cameras Tests Conclusions Stability Range Uniformity Scene Spatial Res. Thermal Res.

Quality ? ! X Effect on Examinations OK Introduction Protocol Quality Assurance of TI Quality ? Introduction Protocol IR Cameras Tests Conclusions Effect on Examinations Examination offset drift fluct. unif. flood resol Qualitative, intra-image A1<B1 OK ! Qualitative, inter-image A1<A2 Quantitative, intra-image A1 - B1 = X°C X Quantitative, inter-image A1 - A2 = X°C Stability Range Uniformity Scene Spatial Res. Thermal Res. What now?

New Approach ! X ! Multi Point Calibration OK OK Introduction Protocol Quality Assurance of TI New Approach Introduction Protocol IR Cameras Tests Conclusions Multi Point Calibration later at 15:00…. Examination offset drift fluct. unif. flood resol Qualitative, intra-image A1<B1 OK ! Qualitative, inter-image A1<A2 Quantitative, intra-image A1 - B1 = X°C X Quantitative, inter-image A1 - A2 = X°C Examination offset drift fluct. unif. flood resol Qualitative, intra-image A1<B1 OK ! Qualitative, inter-image A1<A2 Quantitative, intra-image A1 - B1 = X°C Quantitative, inter-image A1 - A2 = X°C Stability Range Uniformity Scene Spatial Res. Thermal Res.