1 Motivation Problem: Amateur photographers take unappealing pictures (e.g. personal and business use) Help users take better pictures with digital cameras Solution: Improve composition during image acquisition Detect main subject in the picture Detect distracting background object (avoidance of mergers) Avoid Merger Amateur Professional

2 Mitigation of Mergers: Overview Goal: Identify background objects merging with main subject In-focus background object Connected to main subject mask Large area relative to image size Merger detection Color segmentation based on hue Identify distracting background object based on distance to main subject and frequency content Blur merging background objects to induce a sense of distance Merging background objects: trees and bush over right shoulder

3 Segmentation of Background Objects Hues above histogram average are dominant hues Background is a mixture of dominant hues Thresholds: average of two consecutive dominant hues Background hues Histogram of background hues and identified objects Thresholds = {87, 151}

4 Merger Object Detection Define Frequency Inverse Distance Measure for each disjoint background object O i Decreases with distance (d i ) from main subject mask Increases with high spatial frequency coefficients (w i H ) Merged object: Object with highest measure

5 Measure Selection Linear, division, and exponential forms to combine High frequencies from residual in Gaussian pyramid Euclidean distance measured from main subject mask AttributeLinearDivisionalExponential Computational complexityLowHigh Merged object’s sizeLargeSmall Divisional Exponential Linear

6 Merger Mitigation Results Background tree and bush merging with main subject High frequency and inv. distance values for background Blurred tree and bush appear to be farther away

7 Per-pixel Implementation Complexity Process /OperationMultiply- accumulates ComparesMemory accesses RGB to hue364 Histogram and thresholding12 RGB to intensity2 Gaussian pyramid94 Approx. inv. distance measure212 Detect merged object11 Gaussian pyramid reconstruction915 TOTAL For comparison, JPEG compression takes ~60 operations/pixel

8 Merger Mitigation System Prototype Merger mitigated picture Binary main subject mask Intensity Gaussian pyramid Background segmentation Inverse distance transform Grayscale image X Detect merging object Grayscale image Reconstruct color pyramid Color Gaussian pyramid Transform coefficients Original color image

9 Conclusion Contributions Combine optical and digital image processing for improved image acquisition Provide online feedback to amateur photographers Mitigation of mergers with background objects Amenable to fixed-point implementation in digital still cameras Independent of scene setting or content Deliverables Prototype development for digital still image acquisition Copies of MATLAB code, slides, and papers, available at

10 ION FOR IMPROVED ACQUISTION Serene Banerjee and Brian L. Evans Embedded Signal Processing Laboratory Wireless Networking and Communications Group IN-CAMERA MERGER MITIGAT

11 IN DIGITAL STILL CAMERAS