1. Quality Assurance in Thermography
P. Plassmann
C.D. Jones
E.F.J. Ring
University of Glamorgan, Faculty of Advanced Technology, UK
R. Simpson
National Physical Laboratory, Radiation Thermometry Group, UK

2. Introduction The rise, fall & rise of Thermography Another fall ahead?
Quality Assurance of TI
Introduction
Introduction
Protocol
IR Cameras
Tests
Conclusions
The rise, fall & rise of Thermography
Another fall ahead?
Reasons?
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
“Thermography” MedLine/Ovid citations

3. Introduction Past Reasons for Decline Future Threats !
Quality Assurance of TI
Introduction
Past Reasons for Decline
Future Threats !
Thermography cover in US (Medicare) removed in 1990s
Poor sensor quality
Poor image processing techniques
Inadequate physiological knowledge
Expectations too high
Indiscrete use and improper interpretation with no standardised protocols
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
Indiscrete use and improper interpretation with no standardised protocols

4. Introduction No Standardisation Means
Quality Assurance of TI
Introduction
Introduction
Protocol
IR Cameras
Tests
Conclusions
No Standardisation Means
No reliable, quantifiable measurements
Interpretation of results difficult
Multi-centre trials unreliable
Longitudinal trials difficult
Acceptance of technique limited
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
Public image of medical thermographer

5. Protocol To Measure = To Compare
Quality Assurance of TI
Protocol
To Measure = To Compare
Standardisation
Quality control
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
Therefore standardisation and quality control of
IR Cameras
Patient
Imaging
Analysis
Storage
File Exchange
Presentation
Protocols

6. Imagers Variety of Systems…. ….Common Problem:
Quality Assurance of TI
Imagers
Introduction
Protocol
IR Cameras
Tests
Conclusions
Variety of Systems….
….Common Problem:
Not designed for exact quantification in the medical range of temperatures
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

7. 4°C is 20% of human skin temperature range
Quality Assurance of TI
Manufacturer’s Specifications
Introduction
Protocol
IR Cameras
Tests
Conclusions
Accuracy (bias, offset)
Most imagers have ± 2°C offset
(Do you know how accurate yours is?)
This is after Calibration
(When was yours calibrated last?)
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
4°C is 20% of human skin temperature range
Resolution
Spatial (128x128 or better) meaning?
Thermal (most better 0.1°C) meaning?

8. Tests Stable Reference: Ice & Water Emissivity close to that of skin
Quality Assurance of TI
Tests
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stable Reference: Ice & Water
Emissivity close to that of skin
Ice + water (almost) exactly 0º C
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
grey: ice, black: water

9. Stability Offset Drift
Quality Assurance of TI
Stability
Introduction
Protocol
IR Cameras
Tests
Conclusions
Offset Drift
Cameras tested against UK’s National Physical Laboratory calibration source
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

10. Stability Offset ‘wobble’ due to Internal Calibration Shutter
Quality Assurance of TI
Stability
Offset ‘wobble’ due to Internal Calibration Shutter
FLIR SC500 (before repair)
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
How to Test?

11. Stability TEST Preparation Room T stable (usual examination setting)
Quality Assurance of TI
Stability TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Preparation
Room T stable (usual examination setting)
Set camera emissivity to 0.98 (for 3-5µm cameras) or 0.99 (for 9-12µm cameras)
Distance ~ 1m
Narrow T range around zero ºC (e.g. -5 to +5)
Switch camera off, allow it to acclimatise to room temperature for at least one hour.
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

12. Long-Term & Short-Term Stability Tests similar
Quality Assurance of TI
Stability TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Method
Fill bucket with ice/water mix, insulate
Camera perpendicular to ice/water
Switch camera on
Stir bucket, baseline image of the ice/water
Take images every 5 minutes (during the first 30 minutes, every 10 minutes after that) for a total of 2 hours. (Stir bucket before each image)
Plot results
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
Long-Term & Short-Term Stability Tests similar

13. Range Offset neither Constant nor Linear ‘Human’ temperature range
Quality Assurance of TI
Range
Introduction
Protocol
IR Cameras
Tests
Conclusions
Offset neither Constant nor Linear
‘Human’ temperature range
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
Do you know yours?

14. Range TEST Preparation Room T stable (usual examination setting)
Quality Assurance of TI
Range TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Preparation
Room T stable (usual examination setting)
Set camera emissivity to 0.98 (for 3-5µm cameras) or 0.99 (for 9-12µm cameras)
Distance ~ 1m
Switch camera on, allow it to settle (use data from ‘Stability’ experiment).
T range 20ºC to 40ºC
Container of water at about 40º C
Platinum resistance or other calibrated thermometer required (ideally traceable to ITS90)
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

15. Range TEST Method Stir it for at least 1 minute (use thermometer)
Quality Assurance of TI
Range TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Method
Stir it for at least 1 minute (use thermometer)
Take 1st image + thermometer readout
While the water is cooling, take images and thermometer readings every 0.5 ºC or so.
Always stir the water for 1 minute before taking a reading/image.
Add small amounts of cold water to speed-up the cooling.
Plot result
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

16. Uniformity Offset Variations due to Optical Limits
Quality Assurance of TI
Uniformity
Introduction
Protocol
IR Cameras
Tests
Conclusions
Offset Variations due to Optical Limits
Corners most problematic
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
T=1.2°C
Ever tested yours?

17. Uniformity TEST Preparation Room T stable (usual examination setting)
Quality Assurance of TI
Uniformity TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Preparation
Room T stable (usual examination setting)
Set camera emissivity to 0.98 (for 3-5µm cameras) or 0.99 (for 9-12µm cameras)
Distance ~ 1m, but de-focus max.
Switch camera on, allow it to settle (use data from ‘Stability’ experiment).
Wide diameter bucket or tray of water (water surface to entirely fill field of view)
T ~ 23ºC (ca. mid range skin temperature)
Narrow T range (21 to 25º C)
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

18. Uniformity TEST Method Stir it for 30 seconds
Quality Assurance of TI
Uniformity TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Method
Stir it for 30 seconds
Take an image (water surface only)
Analyse image
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

19. How does your Imager behave?
Quality Assurance of TI
Scene
Introduction
Protocol
IR Cameras
Tests
Conclusions
Offset Variations due to ‘Flooding’
Un-cooled imagers most at risk
30.19°C
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
30.32°C
35.29°C
T=0.13°C
How does your Imager behave?

20. Scene TEST Preparation Room T stable (usual examination setting)
Quality Assurance of TI
Scene TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Preparation
Room T stable (usual examination setting)
Set camera emissivity to 0.98 (for 3-5µm cameras) or 0.99 (for 9-12µm cameras)
Distance ~ 1m
Switch camera on, allow it to settle (use data from ‘Stability’ experiment).
Bucket of water at room T
Bucket of water at ~ 40ºC
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

21. Scene TEST Method Take image 1 (room T water surface only)
Quality Assurance of TI
Scene TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Method
Take image 1 (room T water surface only)
Take image 2 (room T water + part of 40º C surface)
Analyse
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
30.32°C
30.19°C

22. Spatial Resolution Temperature Peak Readout Error
Quality Assurance of TI
Spatial Resolution
Introduction
Protocol
IR Cameras
Tests
Conclusions
Temperature Peak Readout Error
Establish minimum object size for correct temperature readout
Similar to MFOV ‘split response’ method (Measurement Field of View)
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

23. Spatial Resol. TEST Preparation
Quality Assurance of TI
Spatial Resol. TEST
Preparation
Room T stable, check emissivity, camera settled (as before)
Distance ~ 30cm
Bucket of water at ~ 40ºC
Cardboard with  cutout (10cm x 2 cm, aluminium markers)
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

24. Spatial Resol. TEST Method
Quality Assurance of TI
Spatial Resol. TEST
Method
Take image (note distance camera - cardboard, d1)
Introduction
Protocol
IR Cameras
Tests
Conclusions
produce cross-sections
3.1 cm z
z = = 6.2mm
2cm x 3.1cm 10cm
calc. width of “just about” cross-section, z
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
calculate width wmin for other distances d2
wmin=
z x d2 d1
10 cm
2 cm

25. Thermal Resolution Noise + Digitisation Step Error
Quality Assurance of TI
Thermal Resolution
Introduction
Protocol
IR Cameras
Tests
Conclusions
Noise + Digitisation Step Error
Usual measure: NETD (Noise Equivalent Temperature Difference), impractical:
varies over time
requires specialised equipment
not standardised (ΔT 5°C or 10°C, object temp…)
Instead: ThRes = digit.Step “+” noise
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
Digitisation Step Example:
Measurement range: 25°C, Digitisation: 8 bit (256 steps)
1 digit step = 25°C / °C
pixels
28.1°C
28.2°C
28.3°C
cross-section

26. Std. Deviation TEST Preparation Method bucket of water, T ~ 30ºC
Quality Assurance of TI
Std. Deviation TEST
Introduction
Protocol
IR Cameras
Tests
Conclusions
Preparation
bucket of water, T ~ 30ºC
rest the same as under ‘Uniformity’
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
Method
Take image (camera de-focused max.)
Calculate Standard Dev. min. AOI 10 x 10 pixels

27. Std. Deviation TEST Method (cont)
Quality Assurance of TI
Std. Deviation TEST
Method (cont)
Calculate Thermal Resolution (95% confidence)
ThRespix = digStep² + (2 x std.Dev)²
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
real T2
std.dev.
digStep
real T1
Example (95% confidence):
ThRespix = ² +4 x 0.14² = 0.3 °C

28. Thermal Resolution Consequences Don’t use spot measurements !
Quality Assurance of TI
Thermal Resolution
Consequences
Don’t use spot measurements !
Always use AOIs
error halves with double amount of pixels squared
therefore:
1 pixel: ± 0.3°C
4 pixels: ± 0.15°C
16 pixels: ± 0.075°C
64 pixels: ± °C
256 pixels: ± °C
8 bits per pixel are sufficient
manufacturer quoted resolution not that important
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

29. Conclusion (1 of 2) Current IR Imagers are Deficient Specifications
Quality Assurance of TI
Conclusion (1 of 2)
Current IR Imagers are Deficient
Specifications
- offset of ±2°C even after manufacturer calibration
Stability
- short-term drift after internal calibration possible (±0.3°C)
- medium term, even after switch-on time specified by
manufacturer, (±1°C)
- long-term, over several hours/days (±0.5°C)
Range
- fluctuation of around ±1°C possible within ‘human’ range
Uniformity
- optical limitations typically ±0.5°C
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

30. Conclusion (2 of 2) Current IR Imagers are Deficient Scene
Quality Assurance of TI
Conclusion (2 of 2)
Current IR Imagers are Deficient
Scene
- flooding effect ±0.2°C
Spatial Resolution
- border / slit effect (not quantifiable, just don’t do it)
Thermal Resolution
- digitisation ‘+’ noise effect ±0.3°C
Introduction
Protocol
IR Cameras
Tests
Conclusions
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.

31. Quality ? ! X Effect on Examinations OK Introduction Protocol
Quality Assurance of TI
Quality ?
Introduction
Protocol
IR Cameras
Tests
Conclusions
Effect on Examinations
Examination
offset
drift
fluct.
unif.
flood
resol
Qualitative, intra-image
A1<B1 OK !
Qualitative, inter-image
A1<A2
Quantitative, intra-image
A1 - B1 = X°C X
Quantitative, inter-image
A1 - A2 = X°C
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.
What now?

32. New Approach ! X ! Multi Point Calibration OK OK Introduction Protocol
Quality Assurance of TI
New Approach
Introduction
Protocol
IR Cameras
Tests
Conclusions
Multi Point Calibration
later at 15:00….
Examination
offset
drift
fluct.
unif.
flood
resol
Qualitative, intra-image
A1<B1 OK !
Qualitative, inter-image
A1<A2
Quantitative, intra-image
A1 - B1 = X°C X
Quantitative, inter-image
A1 - A2 = X°C
Examination
offset
drift
fluct.
unif.
flood
resol
Qualitative, intra-image
A1<B1 OK !
Qualitative, inter-image
A1<A2
Quantitative, intra-image
A1 - B1 = X°C
Quantitative, inter-image
A1 - A2 = X°C
Stability
Range
Uniformity
Scene
Spatial Res.
Thermal Res.