Download presentation
Presentation is loading. Please wait.
Published byEsmond Hall Modified over 9 years ago
1
Photo-Quality Enhancement based on Visual Aesthetics S. Bhattacharya*, R. Sukthankar**, M.Shah* *University of Central Florida, **Intel labs & CMU
2
Motivation
3
Related work Quality enhancement framework Visual aesthetics Aesthetic appeal assessment Enhancement through recomposition Experimental results Conclusions Future directions Outline
4
Low-level (dehazing etc.) Related Work Domain-specific (face beautification etc.) T. Leyvand,et al., “Data-Driven Enhancement of Facial Attractiveness”, ACM SIGGRAPH 08 K. He, J.Sun X. Tang, “Single Image Haze Removal using Dark Channel Prior”, CVPR 09
5
Overview Enhanced Image Input Image Enhancement Engine Assessment Engine Aesthetic Features Image Semantics, Aesthetic Features Appeal Prediction Recomposition Aesthetic Model
6
Visual Aesthetics: Rule of Thirds Motivated by Renaissance Paintings… Rule of thirds: Subject of interest is aligned to one of the stress points Professional photographs also abide this: http://howtophotography.org/wp-content/uploads/2010/06/rule-of-thirds-photo2.jpg http://hoocher.com/Joseph_William_Turner/Joseph_William_Turner.htm
7
Visual Aesthetics: Golden Ratio http://hoocher.com/Joseph_William_Turner/Joseph_William_Turner.htm Divine proportion: Horizon divides sky and sea/land according to golden ratio. http://www.dptips-central.com/rules-of-composition.html An example professional photographic composition: ~1.618k Sky Sea Sky Land k
8
Single subject Compositions (384) Modeling Aesthetics: Dataset Landscapes/Seascapes (248) http://www.flickr.com
9
Single subjects Modeling Aesthetics: User study Landscapes/Seascapes http://www.flickr.com 121514 … Rank Assignment between 1-5 Ground Truth Appeal Factors
10
Modeling Aesthetics: User study 1.76 4.23 Poorly rated images Best rated images
11
Modeling Aesthetics: User study Appeal Factor Intervals User Agreements Good Compositions Poor compositions
12
Modeling Aesthetics: Features (a) Relative Foreground Location (Rule of Thirds) Visual Attention Center Stress Point
13
Modeling Aesthetics: Features (b) Visual weight deviation from Golden Ratio (Divine Proportion) YkYk YgYg
14
Experiments (Assessment) Learn Support Vector Regression models Prediction accuracy: ◦ Single subject compositions ~ 87% ◦ Landscapes/Seascapes ~ 91% Smooth mapping between Appeal factor and Aesthetic Features Relative Foreground Location Visual Weight Deviation
15
Spatial Recomposition
16
Why Cropping does not work? Optimal Crop
17
Recomposition: Algorithm I Input Image Labeled Elements Semantic Segmentation Single Subject? Optimal Object Placement Spatial Recomposition In-painting
18
Semantic Segmentation Input Image Geometric Context Classifier* *D. Hoiem, A.A. Efros, and M. Hebert, "Geometric Context from a Single Image", ICCV 2005 Post Processing Sky Support Horizon Segmented Foreground
19
Optimal Object Placement Find x that Maximizes Appeal Intensity Term Labeled Image Support Neighborhood Gradient Term s.t. neighbors stay “like neighbors” +
20
Optimization (Example) PAF = 3.31PAF = 3.68 Semantic constraint prevents this PAF = 3.22 Original Image PAF = 4.53 Optimal Solution X
21
Perspective Scaling Scaling Factor Vanishing Point Optimal location Visual Attention Center Scaled Foreground
22
Inpainting Foreground Hole Inpaint Hole Yunjun Zhang. Jiangjian Xiao. Mubarak Shah, “ Region Completion in a Single Image”, EUROGRAPHICS 04
23
Recomposition: Algorithm 2 Input Image Labeled Elements Semantic Segmentation Land/Sea scape? Visual Weight Balancing Optimally Crop/Expand
24
Ratio of Current extents Balancing Visual Weights h = vertical extent of the balanced image Solve for h (sign of h determines crop/expansion) YkYk YgYg Y k +h YgYg
25
Experimental Results Horse is moved to a more visually pleasing location Scaled appropriately Appeal increases by 64% Single Subject Composition Before RecompositionAfter Recomposition
26
Results BeforeAfter PAF = 2.45 PAF = 4.29 PAF = 3.98PAF = 4.46
27
Results BeforeAfter PAF = 3.13PAF = 4.19 PAF = 4.02PAF = 4.34
28
Results Before After PAF = 3.77PAF = 4.25 PAF = 3.92PAF = 4.11
29
Results BeforeAfter PAF = 4.06 PAF = 4.68 PAF = 2.71PAF = 3.26
30
Optimally cropped support region to increase weights for sky Appeal factor increased by 51% Visual weight balancing Results Before RecompositionAfter Recomposition
31
Balancing Visual weights BeforeAfter PAF = 3.83 PAF = 4.02 PAF = 3.92 PAF = 4.38
32
Balancing Visual weights BeforeAfter PAF = 4.02 PAF = 4.71 PAF = 4.17 PAF = 4.49
33
Not Perfect Algorithm says nice, humans: otherwise PAF = 2.34Fa = 2.41 (Ground Truth) Before PAF = 3.63Fa = 2.54 (Ground Truth) After
34
Summary: Optimal Placement BeforeAfter Increased # of Highly rated Images Decreased # of Poorly rated Images
35
Summary: Visual Weights BeforeAfter Increased # of Highly rated Images Decreased # of Poorly rated Images
36
Conclusion Intelligent photo recomposition Can also be used for aesthetic filtering Easy to use practical tool
37
Future Work Synthesizing ideal image from many photos of the same scene Recomposition for videos
38
Questions?
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.