LYU 0602 Automatic PhotoHunt Generation1 Automatic PhotoHunt Generation Shum Hei Lung To Wan Chi Supervisor: Prof. Michael R. Lyu

LYU 0602 Automatic PhotoHunt Generation2 Background Objectives Previous Work Newly Developed Module –Image Analysis –Image Warping –Object Appending Enhanced Module –Elimination –Game Engine Evaluation Conclusion Agenda

LYU 0602 Automatic PhotoHunt Generation3 Background PhotoHunt is … –A Spot-the-difference game –Classic yet evergreen –Popular in electronic game centers all over the world However… It is limited by man power

LYU 0602 Automatic PhotoHunt Generation4 Objectives Develop real-time Image Generation Engine –Employ image processing techniques –Mimic human behavior Develop PhotoHunt game –Make use of generation engine –Implement more unique features

LYU 0602 Automatic PhotoHunt Generation5 Objectives Image Generation Engine To generate an image for PhotoHunt game –Effects that may be applied: Elimination Image Warping Object Appending Color Change Definition of well generated image: The effects should be “NOT OBVIOUS YET DISCOVERABLE”

LYU 0602 Automatic PhotoHunt Generation6 Previous Work Applications Semi Automatic PhotoHunt Game Engine Automatic PhotoHunt Generation Image Generation Engine Segmentation Module Modification Module -Elimination -Color change Smoothing module Image Processing Foundation

LYU 0602 Automatic PhotoHunt Generation7 Gaussian Pyramid Previous Work Segmentation Module Elimination ModuleSmoothing Module To detect and extract segment from the input image Three Phases: –Pyramid Segmentation –Constraint Checking –Reference image building Game Engine

LYU 0602 Automatic PhotoHunt Generation8 Previous Work Segmentation Module Elimination ModuleSmoothing Module Game Engine Direct Copy Algorithm Horizontal Gradient Algorithm Nearest Boundary Algorithm Enhanced Nearest Boundary Algorithm

LYU 0602 Automatic PhotoHunt Generation9 Previous Work Segmentation Module Elimination ModuleSmoothing Module Game Engine To reduce noise and distortion To make the image more realistic Gaussian Filter (Neighbor size=3, sigma=1)

LYU 0602 Automatic PhotoHunt Generation10 Previous Work Segmentation Module Elimination ModuleSmooth Module Game Engine

LYU 0602 Automatic PhotoHunt Generation11 Last semester Segmentation Module Modification Modules Smooth Image Game Engine

LYU 0602 Automatic PhotoHunt Generation12 This Semester Segmentation Module Enhanced Elimination Smooth Image Enhanced Game Engine Image Analysis Modules Image WarpingColor Change Object Appending

LYU 0602 Automatic PhotoHunt Generation13 Image Analysis Module

LYU 0602 Automatic PhotoHunt Generation14 Image Analysis Module Purpose –To extract useful information from the image in order to assist the generation process Segmentation ModuleImage Analysis Module Color Change Elimination Image Warping

LYU 0602 Automatic PhotoHunt Generation15 Image Analysis Module > Function 1 Screening out undesirable segment Definition of undesirable segment –Regions that are wrongly segmented Cause of undesirable segment –The engine uses an optimized threshold to segment all images Assumption Segment & Surrounding have similar color and brightness Undesirable Segment Reject Segment Come from Same Object

LYU 0602 Automatic PhotoHunt Generation16 Image Analysis Module Functions: Screening out undesirable segment Deciding modification to be applied Providing suggestion on replacement color

LYU 0602 Automatic PhotoHunt Generation17 Screening out undesirable segment Image Analysis Module Screening out undesirable segment Procedures: 1.Compute Mean of Object –(1) 2.Compute Mode of Background –(2) 3.Compare (1) and (2) Object Mean : [ ] T Bg Mode : [ ] T Diff : 475>60 Accept Object Mean : [ ] T Bg Mode : [ ] T Diff : 284>60 Accept Object Mean : [ ] T Bg Mode : [ ] T Diff : 32<60 Object Mean : [ ] T Bg Mode : [ ] T Diff : 39<60 Reject

LYU 0602 Automatic PhotoHunt Generation18 > Function 2 Deciding modification to be applied Image Analysis Module > Function 2 Deciding modification to be applied Segment Color Change Image Warping Elimination Single Colored Regular in shape Any shape The property of image for the specified effect:

LYU 0602 Automatic PhotoHunt Generation19 > Function 2 Deciding modification to be applied Image Analysis Module > Function 2 Deciding modification to be applied Object Neighbor Difference<Threshold1 Object Variance < Threshold2 Color Change yes No Object occupied Area>70% yes No Image Warping Elimination

LYU 0602 Automatic PhotoHunt Generation20 Deciding modification to be applied

LYU 0602 Automatic PhotoHunt Generation21 > Function 3 uggestion on replacement color Image Analysis Module > Function 3 Suggestion on replacement color ParameterMeaning Background Neighbor Difference The complexity of background Background ModeThe major color of background Background MeanThe texture property of the background

LYU 0602 Automatic PhotoHunt Generation22 Image Warping Module

LYU 0602 Automatic PhotoHunt Generation23 Image Warping Produce Distortion by applying geometric transformation.

LYU 0602 Automatic PhotoHunt Generation24

LYU 0602 Automatic PhotoHunt Generation25 Image Warping Forward mapping algorithm Transformation1 Quadratic of y Transformation2 linear of x Transformation Equation (General)

LYU 0602 Automatic PhotoHunt Generation26 Transformation 1 Transformation 2 Transformation Equation minX Δy=0 Mid pt Δy=max maxX Δy=0 Δy max 1.Quadratic equation of root x=minX or maxX 2. Substituting (midptX, Δy max ) into the equation to acquire the weight to control curvature 1.Flip the upper part, y>mid pt y 2.Enlarge the curve with a ratio proportional to distance between mid pt of y maxY minY midptY

LYU 0602 Automatic PhotoHunt Generation27

LYU 0602 Automatic PhotoHunt Generation28 Object Appending Module

LYU 0602 Automatic PhotoHunt Generation29 Object Appending To append an object from our database to the original image Unable to carry out object recognition –Only generic objects are inserted to engine

LYU 0602 Automatic PhotoHunt Generation30 Examples:

LYU 0602 Automatic PhotoHunt Generation31 Enhanced Elimination Module

LYU 0602 Automatic PhotoHunt Generation32 Elimination Module Hybrid Elimination Makes use of statistic data that came from the image analysis module Information Needed: –Background Mode –Background Neighbor Difference –Background Mean

LYU 0602 Automatic PhotoHunt Generation33 Hybrid Elimination Algorithm Check Background Neighbor Difference - To check whether the background is single colored Case 1: Use the Background Mode to replace Case 2: Apply texture from surrounding –Select the suitable surrounding region –Apply Direct copy Algorithm

LYU 0602 Automatic PhotoHunt Generation34 Game Engine

LYU 0602 Automatic PhotoHunt Generation35 System Overview

LYU 0602 Automatic PhotoHunt Generation36 System Interface In later demo session

LYU 0602 Automatic PhotoHunt Generation37 Evaluation

LYU 0602 Automatic PhotoHunt Generation38 Acceptance Rate Assume Acceptance rate= 75% = 988/1318 accepted images Thus, x-axis is about 200 in upper limit and 4500 in lower limit.

LYU 0602 Automatic PhotoHunt Generation39 Processing Time Run one set of 1318 images for 17 times Average Processing time per set: – 00:15:50 Average Processing time per image – second

LYU 0602 Automatic PhotoHunt Generation40 Image Quality Survey Result Total Visitors : 91 Total Hits: 1109 Total Images : 264 Received Survey : 121 Total Segments : 605 Satisfy 76% Not Satisfy 24% Characteristics image shared for achieving good effects: Many objects within the image Sharp Edge of object Less noise Maybe a cartoon

LYU 0602 Automatic PhotoHunt Generation41 Demo

LYU 0602 Automatic PhotoHunt Generation42 Conclusion Developed the image generation engine Developed the game engine Carried out testing and analysis on the system Published the product to the public We are still watching the statistic and looking at feedback to improve our system

LYU 0602 Automatic PhotoHunt Generation43 Q & A

LYU 0602 Automatic PhotoHunt Generation44 The End Thanks for your kind attention.