1. Towards Clinically-relevant Standardization of Image Quality
Ehsan Samei, Duke University
Alan Rowberg, University of Washington
Ellie Avraham, Eastman Kodak Company
Craig Cornelius, Eastman Kodak Company
22-Sept-2003

2. Objectives
Describe current medical image quality and consistency performance efforts
Identify limitations in existing standards
Outline 3 specific proposals:
Add new image quality factors to standards
Update DICOM IQ performance services
Research technical-clinical connection
22-Sept-2003

3. DICOM Image Consistency Efforts
Three DICOM initiatives ( ):
Grayscale Standard Display Function (GSDF)
Presentation LUT (P-LUT)
Grayscale Softcopy Presentation State (GSPS)
And in actual practice…
Are widely & effectively implemented
Via IHE Consistent Presentation of Images solution:
Promoted conformance testing
Demonstrated inter-vendor consistency
22-Sept-2003

4. DICOM GSDF: Barten CurveL2
~150 cd/m2 J2
Absolute luminance increment
L1 << L2 L1
~3 cd/m2 J1
Number of perceptual levels J1 = J2
22-Sept-2003

5. AAPM Task Group 18 Efforts
Team of academic, clinical, and industry contributors
“Assessment of Display Performance for Medical Imaging Systems” (2002):
Practical guidelines for qualitative & quantitative display assessment
Includes all key aspects of display performance
Defines IQ test patterns and procedures
Recommends specific IQ acceptance criteria
22-Sept-2003

6. Standardization gaps DICOM GSDF: AAPM TG18 report: Pros: Pros:
Mathematical definition based on Human Visual System model
Limitations:
Tonescale consistency only: no other IQ factors
No acceptance criteria / conformance procedures
Only for grayscale images
No Display Device Services (Capabilities)
AAPM TG18 report:
Pros:
Provides Professional recommendations
Covers all key display performance aspects
Acceptance Criteria
Quantitative measures
Limitations:
Only guidelines
Not a “standard”
22-Sept-2003

7. The technical-clinical gap:
The connection between quantifiable IQ metrics & clinical performance is unknown:
Luminance: deviations from GSDF, number of gray levels displayed
Spatial: resolution, noise, geometric distortion
Chromaticity variations
Environmental: ambient light, glare, reflection, …
Amount of acceptable variation is unknown
22-Sept-2003

8. 3 proposals to bridge the gaps:
Extend DICOM standard beyond luminance response
Add and update DICOM Service Classes for image quality / performance
Promote research on clinical - technical image quality relationship
22-Sept-2003

9. 1. Display Image Quality (DIQ) Initiative
Add measurable and quantifiable elements of AAPM display performance procedures
Include testing methodologies & defined limits for clinical / diagnostic performance
Quantify visual performance using
Simple test images
Specific observer protocols
Relative acceptance indicators
For both softcopy and hardcopy presentation
22-Sept-2003

10. DIQ Softcopy Examples Quantify % deviation from GSDF curve
Define criteria for min & max luminance
Define visual luminance evaluation
Evaluate specific image quality factors:
Ambient light limits: specular and diffuse
Spatial resolution with TG18-QC/CX
Check geometric distortion with TG18-QC
22-Sept-2003

11. Contrast response comparison
GSDF contrast +/- 10%
Non-standardized display contrast
22-Sept-2003

12. AAPM TG18 Patterns TG18-MP: bit-depth / continuous grayscale
TG18-CT:contrast / luminance response
22-Sept-2003

13. Comprehensive TG18-QC
The comprehensive TG18-QC test pattern for evaluation of key display characteristics:
Resolution
Luminance
Geometric distortion
22-Sept-2003

14. Other DIQ Extensions Hardcopy quality metrics: Color extensions:
Media & printer quality (e.g., visible coating variations, distortions, artifacts)
GSDF compliance, # of JNDs theory vs. actual
Printable matrix size
Spatial frequency response fidelity
Color extensions:
Standardize for grayscale areas of color images
Add descriptions for color image characterization
Color display and print device calibration
22-Sept-2003

15. 2. DICOM Service Extensions
Add Display Performance Service Class
Query image quality / performance information
Control / configure manageable settings
Include new IQ factors, e.g., MTF, ambient, …
Extend Printer Configuration Retrieval Service Class
Include access to additional IQ factors
Note: Measurables include both human-evaluated and automatically-measured values
22-Sept-2003

16. Use cases: Standardizing Output
Printing application
Retrieves matrix size and MTF of film printer
Determines type of magnification, if any, to be applied to the image for smallest artifact
Display Performance Service Class User
Requests the luminance characteristic curve from its workstation’s display system
Determines if the display is standardized
If needed, computes an internal image tonescale correction, producing GSDF standardized result
22-Sept-2003

17. Real device  standardized performance
300.00
250.00
Target GSDF
200.00
Actual device
150.00
Luminance
Desired
output
100.00
Modified Input
345
50.00
Original Input
443
0.00
100
200
300
400
500
600
700
Digital Value
22-Sept-2003

18. Use Case: Quality Control Management
Centralized management application:
Queries devices for calibration date, luminance characteristics, ambient light settings, etc.
Records the results in a central database
Creates maintenance lists for displays and printers
Reporting application uses database for:
Regulatory and management reports
Stability and lifetime statistics on displays to support replacement schedules and budgets
22-Sept-2003

19. Network-wide Quality Control
Diagnostic display S
ummary
data
Administrator’s
Summary
data
Film
Printer
Characteristic curve, MTF,
other performance measures
Request
Response
Characteristic curve, MTF,
other performance measures
Web -
based
displays
Modality Capture
console
Clinical display
Reflective Hardcopy C
linical display
Regulatory Reports
Performance
& Lifetime Statistics
22-Sept-2003

20. More use cases… Consultation: Capture Consoles:
Maximize perceptual similarity to ensure “What I see is what you will get!”
Display performance information gives confidence
Capture Consoles:
“The technologist sees what the doctor will get.”
Reduce errors, retakes, miscommunication
Extend quality control program to consoles
22-Sept-2003

21. What the Radiologist Displays: What the Reviewing Physician Sees:
Consistent Presentation of Images On a Calibrated Display
Area Of
Interest
Annotate
Zoom
Area Of
Interest
With Presentation State
Flip
The Radiologist’s Transformations Are Saved
Original Image
Inconsistent Presentation
On an Uncalibrated Display
Window Level
Original Image
Original Image
Original Image
The Radiologist’s Transformations Are Lost
22-Sept-2003

22. 3. Clinical Significant of IQ Measures
Hmm…
We assume there is a connection…
What deviations matter, and how much?
Physical metrics
Clinical performance
22-Sept-2003

23. Clinical impact: unknown
Rendered with DICOM GSDF
Not rendered with DICOM GSDF
22-Sept-2003

24. Goals of proposed research:
Determine the clinical consequence of variations in image quality metrics (e.g., GSDF conformance, MTF, noise, …)
Define what constitutes image quality from a diagnostic perspective
Incorporate results into new joint standards that will utilize standardized test patterns, procedures, and clinical use cases
22-Sept-2003

25. Suggested research approach
Form inter-society committee to:
Design specific research projects
Obtains & review data sets
Solicit & encourage active participation by researchers: radiologists, scientists, …
Arrange reporting of results
Define recommendations to standards and professional groups
Sample research:
Obtain images from 3 radiographic modalities
Present images, simulating nonstandard display behavior
Run observer performance experiments at major professional meetings and events
Analyze by ROC methods
22-Sept-2003

26. Conclusions
Existing standards are insufficient to assure consistent, high quality medical image output.
Steps are proposed to further the reach and impact of DICOM toward quality medicine.
New directions will provide benefits for PACS users, administrators, vendors, and patients.
22-Sept-2003

27. Contact Information
Ehsan Samei, Duke University
Alan Rowberg, M.D., University of Washington,
Ellie Avraham, Eastman Kodak Company,
Craig Cornelius, Eastman Kodak Company,
AAPM Task Group 18 web site:
22-Sept-2003