1. How to Make Printed and Displayed Images Have High Visual Quality
Brian L. Evans
Embedded Signal Processing Laboratory
The University of Texas at Austin
Austin TX
Ph.D. Graduates: Niranjan Damera-Venkata (HP Labs) Thomas D. Kite (Audio Precision)
Graduate Student: Vishal Monga
Support: HP Labs National Science Foundation

2. UT Center for Perceptual Systems
Outline
Introduction
Grayscale halftoning for printing
Screening
Error diffusion
Direct binary search
Linear human visual system model
Color halftoning for display
Optimal design
Conclusion
UT Center for Perceptual Systems

3. Need for Digital Image Halftoning
Many devices incapable of reproducing grayscale (e.g. eight bits/pixel or gray levels from 0 to 255)
Laser and inkjet printers
Facsimile machines
Low-cost liquid crystal displays
Grayscale imagery binarized for these devices
Halftoning tries to reproduce full range of gray while preserving quality and spatial resolution
Screening methods are fast and simple
Error diffusion gives better results on some media
UT Center for Perceptual Systems

4. Digital Halftoning Methods
Clutered Dot Screening
AM Halftoning
Dispersed Dot Screening
FM Halftoning
Error Diffusion
FM Halftoning 1975
Blue-noise Mask FM Halftoning 1993
Green-noise Halftoning
AM-FM Halftoning 1992
Direct Binary Search
FM Halftoning 1992
UT Center for Perceptual Systems

5. Screening (Masking) Methods
Periodic array of thresholds smaller than image
Spatial resampling leads to aliasing (gridding effect)
Clustered dot screening produces a coarse image that is more resistant to printer defects such as ink spread
Dispersed dot screening has higher spatial resolution
Blue noise masking uses large array of thresholds
UT Center for Perceptual Systems

6. Grayscale Error Diffusion
Shape quantization noise into high frequencies
Design of error filter key to quality
Not a screening technique
Error Diffusion
Spectrum + _
e(m)
b(m)
x(m)
difference
threshold
compute error
shape error
u(m)
current pixel
3/16
7/16
5/16
1/16
weights
2-D sigma-delta modulation
UT Center for Perceptual Systems

7. Simple Noise Shaping Example
Two-bit output device and four-bit input words
Going from 4 bits down to 2 increases noise by ~ 12 dB
Shaping eliminates noise at DC at expense of increased noise at high frequency. 4 2
Average output = 1/4( )=1001
Input
words
To output device
4-bit resolution at DC! 2 2
Added noise
1 sample
delay
Assume input = 1001 constant
12 dB
(2 bits)
Time Input Feedback Sum Output f
Periodic
If signal is in this band, you are better off
UT Center for Perceptual Systems

8. Modeling Grayscale Error Diffusion
Sharpening caused by a correlated error image [Knox, 1992]
Floyd- Steinberg
Jarvis
UT Center for Perceptual Systems
Error images
Halftones

9. Modeling Grayscale Error Diffusion
Apply sigma-delta modulation analysis to 2-D
Linear gain model for quantizer in 1-D [Ardalan and Paulos, 1988]
Linear gain model for grayscale image [Kite, Evans, Bovik, 2000]
Signal transfer function (STF), noise transfer function (NTF) {
us(m)
Ks us(m) Ks
Signal Path
Q(.)
u(m)
b(m)
n(m)
Simple linear equalizers are easy to implement but enhance noise
Complex equalizers such as the DFE is required (how many taps has an DFE ?)
The computational complexity of DFE increases very fast with bit rate
un(m)
Kn un(m) + n(m) +
Noise Path
1 – H(z) is highpass so H(z) is lowpass
UT Center for Perceptual Systems

10. Problems with Error Diffusion
Objectionable artifacts
Scan order affects results
“Worminess” visible in constant graylevel areas
Image sharpening
Error filters due to [Jarvis, Judice & Ninke, 1976] and [Stucki, 1980] reduce worminess and sharpen edges
Sharpening not always desirable: may be adjustable by prefiltering based on linear gain model [Kite, Evans, Bovik, 2000]
Computational complexity
Larger error filters require more operations per pixel
Push towards simple schemes for fast printing
UT Center for Perceptual Systems

11. UT Center for Perceptual Systems
Direct Binary Search
Minimize mean-squared error between lowpass filtered versions of grayscale and halftone images
Lowpass filter is based a linear model of the human visual system (contrast sensitivity function)
Iterative method that gives a practical upper bound on the achievable quality for a halftone of an original image
Each iteration visits every pixel
At each pixel, consider changing state of the pixel (toggle) or swapping it with each of its 8 nearest neighbors that differ in state from it
Terminate when if no pixels are changed in an iteration
UT Center for Perceptual Systems

12. UT Center for Perceptual Systems
Direct Binary Search
Advantages
Significantly improved image quality over screening and error diffusion methods
Quality of final solution is relatively insensitive to the choice of starting point (initial halftone)
Has application in off-line design of threshold arrays for screening methods
Disadvantages
Computational cost and memory usage is very high in comparison to error diffusion and screening methods
Increase complexity makes it unsuitable for real-time applications such as printing
UT Center for Perceptual Systems

13. Contrast Sensitivity Function
Contrast needed at a particular spatial frequency for visibility
Angular dependence modeled with a cosine function
Modify it at low frequency to be lowpass, which gives better correlation with psychovisual results
Useful in image quality metrics for optimization and performance evaluation
UT Center for Perceptual Systems

14. Color Halftoning by Error Diffusion for Display
Input image has a vector of values (e.g. Red-Green-Blue) at each pixel
Error filter has matrix-valued coefficients
Algorithm for adapting matrix coefficients [Akarun, Yardimci, Cetin 1997] + _
e(m)
b(m)
x(m)
difference
threshold
compute error
shape error
u(m)
t(m)
The frequency domain equalizer is just a complex division per subchannel
Channel shortening equalizer is an tap FIR filter
My focus is on channel shortening
UT Center for Perceptual Systems

15. Matrix Gain Model for the Quantizer
Replace scalar gain with a matrix
Noise uncorrelated with signal component of quantizer input
Convolution becomes matrix–vector multiplication in frequency domain
u(m) quantizer input b(m) quantizer output
The frequency domain equalizer is just a complex division per subchannel
Channel shortening equalizer is an tap FIR filter
My focus is on channel shortening
Noise component of output
Signal component of output
UT Center for Perceptual Systems

16. Optimum Color Noise Shaping
Vector color error diffusion halftone model
We use the matrix gain model [Damera-Venkata and Evans, 2001]
Predicts signal frequency distortion
Predicts shaped color halftone noise
Visibility of halftone noise depends on
Model predicting noise shaping
Human visual system model (assume linear shift-invariant)
Formulation of design problem
Given HVS model and matrix gain model find the color error filter that minimizes average visible noise power subject to certain diffusion constraints
UT Center for Perceptual Systems

17. Linear Color Vision Model
Pattern-Color separable model [Poirson and Wandell, 1993]
Forms the basis for S-CIELab [Zhang and Wandell, 1996]
Pixel-based color transformation
B-W
R-G
The frequency domain equalizer is just a complex division per subchannel
Channel shortening equalizer is an tap FIR filter
My focus is on channel shortening
B-Y
Opponent
representation
Spatial
filtering
UT Center for Perceptual Systems

18. Linear Color Vision Model
Undo gamma correction on RGB image
Color separation
Measure power spectral distribution of RGB phosphor excitations
Measure absorption rates of long, medium, short (LMS) cones
Device dependent transformation C from RGB to LMS space
Transform LMS to opponent representation using O
Color separation may be expressed as T = OC
Spatial filtering is incorporated using matrix filter
Linear color vision model
The frequency domain equalizer is just a complex division per subchannel
Channel shortening equalizer is an tap FIR filter
My focus is on channel shortening
where
is a diagonal matrix
UT Center for Perceptual Systems

19. Original Image Sample Images and optimum coefficients for sRGB monitor
available at:
Original Image
UT Center for Perceptual Systems

20. UT Center for Perceptual Systems
Floyd-Steinberg
Optimum Filter
UT Center for Perceptual Systems

21. UT Center for Perceptual Systems
Conclusions
Design of “optimal” color noise shaping filters
We use the matrix gain model [Damera-Venkata and Evans, 2001]
Predicts shaped color halftone noise
HVS could be modeled as a general LSI system
Solve for best error filter that minimizes visually weighted average color halftone noise energy
Future work
Above optimal solution does not guarantee “optimal” dot distributions
Tone dependent error filters for optimal dot distributions
Improve numerical stability of descent procedure
UT Center for Perceptual Systems

22. Designing the Error Filter
Backup Slides
Designing the Error Filter
Eliminate linear distortion filtering before error diffusion
Optimize error filter h(m) for noise shaping
Subject to diffusion constraints
where
The frequency domain equalizer is just a complex division per subchannel
Channel shortening equalizer is an tap FIR filter
My focus is on channel shortening
linear model of human visual system
matrix-valued convolution
UT Center for Perceptual Systems

23. Generalized Optimum Solution
Backup Slides
Generalized Optimum Solution
Differentiate scalar objective function for visual noise shaping w/r to matrix-valued coefficients
Write norm as trace and differentiate trace using identities from linear algebra
The frequency domain equalizer is just a complex division per subchannel
Channel shortening equalizer is an tap FIR filter
My focus is on channel shortening
UT Center for Perceptual Systems

24. Generalized Optimum Solution (cont.)
Backup Slides
Generalized Optimum Solution (cont.)
Differentiating and using linearity of expectation operator give a generalization of the Yule-Walker equations
where
Assuming white noise injection
The frequency domain equalizer is just a complex division per subchannel
Channel shortening equalizer is an tap FIR filter
My focus is on channel shortening
Solve using gradient descent with projection onto constraint set
UT Center for Perceptual Systems

25. Implementation of Vector Color Error Diffusion
Backup Slides
Implementation of Vector Color Error Diffusion
Hgr
Hgg +
Hgb
UT Center for Perceptual Systems