1. An automated image prescreening tool for a printer qualification process by † Du-Yong Ng and ‡ Jan P. Allebach † Lexmark International Inc. ‡ School of Electrical and Computer Engineering, Purdue University

2. Synopsis Anatomy of a formatter-based EP laser printer Overview of a printer qualification process Motivation Examples of artifacts Image fidelity metrics – prior work Prescreening tool Experiment Results Conclusion

3. Anatomy of a formatter-based EP laser printer A standalone network device Print Engine Formatter Control Panel

4. Overview of a printer qualification process Phase I  The formatter (hardware + firmware) and the print engine are developed in parallel.  The hardware portion of the formatter is not ready.  The firmware of formatter is tested using a simulator. Test image (PDL) Newly developed firmware + simulator Newly developed firmware + simulator Firmware of an earlier product + simulator Firmware of an earlier product + simulator Master Current Digital outputs match qualitatively?

5. Overview of a printer qualification process (cont.) Phase II  Preproduction print engines are either scarce or not available yet.  Preproduction formatter hardware is available.  The formatter (firmware + hardware) is tested with a print engine emulator. Test image (PDL) Newly developed formatter + print engine emulator Newly developed formatter + print engine emulator Formatter of an earlier product + print engine emulator Formatter of an earlier product + print engine emulator Digital outputs match qualitatively? Master Current

6. Overview of a printer qualification process (cont.) Phase III  Preproduction printers (formatter + print engine) are available. Test image (PDL) Newly developed printer Earlier printer model Hardcopies match qualitatively? Master Current

7. Overview of a printer qualification process - summary Simulator without actual formatter / emulator with actual formatter Test image (PDL) Softcopy current image Comparison masters Softcopy Formatter/ firmware development teams Debug Error flagged Actual formatter Hardcopy current image Print engine Hardcopy Debug

8. Motivations Image screening (comparison of master-current image pairs)  is mostly performed by trained observers.  is needed for thousands of softcopy and hardcopy image pairs throughout the printer development process.  is very labor intensive.  is often performed only on a fraction of test suites before the product is rolled out due to »a relatively short development time. »a large number of test images in the test suites. Our goal  is to develop a automated tool to reduce the workload of trained observers and increase the volume of tests the by screening out softcopy image pairs with »highly objectionable errors (failed). »visually insignificant errors (passed). Trained observers only need to focus on image pairs which could not be screened out by the tool (further evaluation)

9. Examples of master-current image pair Master Current Different halftone algorithms Missing pixels MasterCurrent

10. Examples of master-current image pair (cont.) Different in character size Master Current Sporadic difference Current Master

11. Image fidelity metrics – prior work First category  Examples »Peak-signal-to-noise ratio (PSNR), root mean square error, and ∆E a*b*  Pros »They are fast and easy to compute, and produce a single number.  Cons »Averaging effect destroys local information and spatial interaction of pixels is ignored. Second category  Examples »Wu’s color image fidelity assessor (HVS model) and structural similarity image metric (first and second order statistics the luminance channel of a local window)  Pros »Spatial processing model is included.  Cons »The algorithm is computational expensive and does not produce a single number (Wu’s color fidelity assessor) »SSIM produces a single number but suffers from the averaging effect.

12. Prescreening tool - requirements needs to work reasonably fast. classifies the master-current pairs into one of the categories: ‘passed’, ‘failed’, and ‘further evaluation required’. must adapt automatically as the spatial resolution of the images changes in dpi. has to be capable of processing any image content. needs to handle both halftone and continuous-tone images. will process full color, indexed color, grayscale, and bilevel images consistently. will detect and ignore small spatial shifts in content and differences in orientation

13. Master image Current image PASSED FAILED The prescreening tool FURTHER EVALUATION 13 Preprocessing Compute 2D error map Cluster error map Compute error metric Thresholding

14. Prescreening tool - preprocessing Determine shift in content Master Current Determine image type Perform quick diff Determine resolution Correct orientation Correct spatial shift Other processing Content size Image size Check dimension Rotated CWRotated CCW MasterCurrent Check orientation 14

15. Prescreening tool – compute and cluster 2D error map (illustration) Master M i j i j Current C i j Binary Error Map j i Clustered Error Map Clustering k th cluster (k+1) th cluster (k+2) th cluster

16. Prescreening tool – compute error metric (contrast sensitivity function CSF component) Master i j i j Current correspond to the l th pixel of the k th cluster

17. Average filtered pixel value for the k th cluster  Master :  Current: Error for the k th cluster: CSF error component for the image pair: To CIE L*a*b* Prescreening tool – compute error metric (contrast sensitivity function CSF component) (cont.)

18. Prescreening tool – compute error metric (visual acuity filter VAF component) Master i j i j Current correspond to the l th pixel of the k th cluster

19. Average filtered pixel value for the k th cluster  Master :  Current: Error for the k th cluster: VAF error component for the image pair: To CIE L*a*b* Prescreening tool – compute error metric (visual acuity filter VAF component) (cont.)

20. Prescreening tool – compute error metric (overall error) Combined error Error metric value for the image pair

21. Experiment There are 147 image pairs. Test image pairs are of  300, 600 and 1200 dpi.  binary, grayscale, color bitmap and full color. Six expert observers classify the image pairs into 3 categories  Acceptable (passed).  Need further evaluation.  Objectionable (failed). These image pairs are processed with our prescreening tool. The peak signal to noise ratio (PSNR) metric and structural similarity image metric (SSIM) are also computed for each preprocessed (to ensure fair comparison) image pair. 21

22. Results – error metric 22 A zero error metric value indicates a perfect match. Decision thresholds exist to pass and fail image pairs. The prescreening tool is able to screen out as many as 35% of the image pairs tested.

23. Results - PSNR A larger PSNR value (PSNR = ∞ for a perfect match) indicates a closer match. Only decision thresholds to pass image pairs exist. The PSNR metric is able to screen out only as many as 4% of the image pairs.

24. Results - SSIM 24 A larger SSIM value (SSIM = 1 for a perfect match) indicates a closer match. Only decision thresholds to fail image pairs exist. The SSIM metric is able to screen out only as many as 3.4% of the image pairs.

25. Conclusion We have successfully developed an automated prescreening tool along with an image fidelity metric for a printer qualification process. This tool works for a wide range of image types, content, and image resolutions. It requires no training and it is able to reduce the workload of expert observers by a substantial amount. 25