LYU 0602 Automatic PhotoHunt Generation Automatic PhotoHunt Generation Shum Hei Lung To Wan Chi Supervisor: Prof. Michael R. Lyu

LYU 0602 Automatic PhotoHunt Generation Background Objectives System Implementation Segmentation Module Elimination Module Other Module Web-based PhotoHunt game Conclusion Overview

LYU 0602 Automatic PhotoHunt Generation Background PhotoHunt is … –a Spot-the-difference game –Classic yet evergreen –Popular in electronic game centers all over the world However… It is limited by man power

LYU 0602 Automatic PhotoHunt Generation Basic techniques to edit image Elimination Color modification Cloning Transformation

LYU 0602 Automatic PhotoHunt Generation Objectives To implement the PhotoHunt Generation Engine –Focus on eliminating object Develop an engine to support real-time image generation for PhotoHunt game

LYU 0602 Automatic PhotoHunt Generation System Implementation

LYU 0602 Automatic PhotoHunt Generation Image Generation Engine To generate an image for PhotoHunt game –Effects that may be applied: Elimination Color Change Duplication Definition of well generated image: The effects should be “NOT OBVIOUS YET DISCOVERABLE”

LYU 0602 Automatic PhotoHunt Generation Image Generation Engine – The Process Segmentation Module Modification Modules Smooth Image GAME Game Engine

LYU 0602 Automatic PhotoHunt Generation Segmentation Module

LYU 0602 Automatic PhotoHunt Generation Segmentation Module To detect and extract segment from the input image Three Phases: 1.Segmentation 2.Constraint Checking Area of Input Image/10000<Area of Segment < Area of Input Image/500 3.Area of Segment Reference image building

LYU 0602 Automatic PhotoHunt Generation Segmentation Algorithm The Segmentation algorithms we have examined: Pixel-based Algorithm Edge Detection Algorithm Region-based Algorithm

LYU 0602 Automatic PhotoHunt Generation Gaussian Pyramid Pyramid Segmentation The Pyramid Segmentation Algorithm Step 1 Generation of the Gaussian pyramid Step 2 Segmentation by pyramid-linking and averaging linked-pixel

LYU 0602 Automatic PhotoHunt Generation Step 1- Generation of the Gaussian pyramid W(-2)W(-1)W(0)W(1)W(2) g(i)Father node Level l-1 Level l

LYU 0602 Automatic PhotoHunt Generation Step 1- Generation of the Gaussian pyramid

LYU 0602 Automatic PhotoHunt Generation Gaussian Pyramid Original photo g0 Dimension= 300*300 Level 1: 150*150 Level 2: 75*75 Level 3: 36*36 Step 1 Computation of the Gaussian pyramid - Step 2 Segmentation by pyramid-linking

LYU 0602 Automatic PhotoHunt Generation Step 2 Segmentation by pyramid-linking and averaging linked-pixel c t (i,j,l) : local characteristics f t (i,j,l) : The potential father node s t (i,j,l) : segment property area t (i,j,l) : area of the segment

LYU 0602 Automatic PhotoHunt Generation 1.Set f t (i,j,l) 2.Initialize area t (i,j,0) =1 c t (i,j,l) =c 0 (i,j,0) 3. for l = 0 to L s t (i,j,L) = c t (i,j,L) for l = L downto 0 s t (i,j,l) = c t (i”,j”,l+1) Step 2 Segmentation by pyramid-linking and averaging linked-pixel

LYU 0602 Automatic PhotoHunt Generation Step 2 Segmentation by pyramid-linking and averaging linked-pixel

LYU 0602 Automatic PhotoHunt Generation Segmentation Result

LYU 0602 Automatic PhotoHunt Generation Elimination Module

LYU 0602 Automatic PhotoHunt Generation Elimination Algorithm Direct Copy Algorithm Horizontal Gradient Algorithm Nearest Boundary Algorithm Enhanced Nearest Boundary Algorithm

LYU 0602 Automatic PhotoHunt Generation Direct Copy Algorithm Directly copy the upper pixel to current pixel Where S i is segmented region of the image for i = 0,1,2, ……

LYU 0602 Automatic PhotoHunt Generation Direct Copy Algorithm

LYU 0602 Automatic PhotoHunt Generation Horizontal Gradient Algorithm Determine color line by line Smoothing the changes of color from one side to another Where C(M) is the color verctor of pixel M and d XY is the distance between X and Y

LYU 0602 Automatic PhotoHunt Generation Horizontal Gradient Algorithm

LYU 0602 Automatic PhotoHunt Generation Horizontal Gradient Algorithm

LYU 0602 Automatic PhotoHunt Generation Horizontal Gradient Algorithm

LYU 0602 Automatic PhotoHunt Generation Nearest Boundary Algorithm Where d min = min(d PN, d QN, d RN, d SN ) Replace the color by the color of nearest boundary

LYU 0602 Automatic PhotoHunt Generation Nearest Boundary Algorithm

LYU 0602 Automatic PhotoHunt Generation Enhanced Nearest Boundary Algorithm Improved from Nearest Boundary Algo. Where d min = min(dPN, dQN, dRN, dSN) S(A,B) is Euclidean distance between A and B

LYU 0602 Automatic PhotoHunt Generation Enhanced Nearest Boundary Algorithm

LYU 0602 Automatic PhotoHunt Generation Enhanced Nearest Boundary Algorithm

LYU 0602 Automatic PhotoHunt Generation Others Modules

LYU 0602 Automatic PhotoHunt Generation Color Change Module Segment dominate color Operation apply RedSwap red and blue channel GreenSwap red and green channel BlueSwap green and blue channel Dominate color segment: % of Pixel of dominate color> 75% Dominate color pixel: DChannel >180 and otherChannel<50

LYU 0602 Automatic PhotoHunt Generation

Smooth Image To reduce noise and distortion To make the image more realistic Gaussian Filter (Neighbor size=3, sigma=1)

LYU 0602 Automatic PhotoHunt Generation Semi-Automatic PhotoHunt Generation

LYU 0602 Automatic PhotoHunt Generation Semi-Automatic PhotoHunt Generation

LYU 0602 Automatic PhotoHunt Generation Game Engine

LYU 0602 Automatic PhotoHunt Generation Game Engine

LYU 0602 Automatic PhotoHunt Generation Game Engine – Front Page

LYU 0602 Automatic PhotoHunt Generation Conclusion

Limitation Only elimination and limited color change effect supported No artificial intelligence control to the Applied effect Modified area found noise and distortion Not all user input can be segmented

LYU 0602 Automatic PhotoHunt Generation Looking Forward… Improve the segmentation algorithm Implement the Duplication and Transformation module Support numbers of object recognitions More Features on the Web-based game Ultimately, to achieve the complete automatic PhotoHunt generation

LYU 0602 Automatic PhotoHunt Generation Q & A

LYU 0602 Automatic PhotoHunt Generation The End Thanks for your kind attention.

LYU 0602 Automatic PhotoHunt Generation AB CD A1 Example of Pyramid Segmentation c 0 (0)= 21 c 0 (1)= 25 c 0 (2)= 2 c 0 (3)= 0 c 0 (4)= 22 c 0 (5)= 15 c 0 (6)= 1 c 0 (7)= 1 c 0 (8)= 13 c 0 (9)= 11 c 0 (10)= 4 c 0 (11)= 5 c 0 (12)= 13 c 0 (13)= 12 c 0 (14)= 12 c 0 (15)= 2 c 0 (A)= 21.2 c 0 (B)= 2.2 c 0 (C)= 14.8 c 0 (D)=3.2 f 0 (0)= A f 0 (1)= A f 0 (2)= B f 0 (3)= B f 0 (4)= A f 0 (5)= C f 0 (6)= B f 0 (7)= B f 0 (8)= C f 0 (9)= C f 0 (10)= B f 0 (11)= D f 0 (12)= C f 0 (13)= C f 0 (14)= C f 0 (15)= D a 0 (A)= 3 a 0 (B)=5 a 0 (C)= 6 a 0 (D)= 2 c 0 (A)= 22.6 c 0 (B)=1.2 c 0 (C)= 12.6 c 0 (D)= 2.5

LYU 0602 Automatic PhotoHunt Generation A3 Pixel-based Segmentation - Thresholding Fig 3.2b Histogram of data generated by ThresSeg.cpp Intensity (0-2 8 ) Frequency Threshold

LYU 0602 Automatic PhotoHunt Generation A4 Edge Detection Segmentation- Canny Edge Detection Phase1 - Gaussian convolution S(x,y)=smoothed image Phase 2 - Edge strength and direction |M|=|H|+|V| Phase 3 - Non-maximum suppression - Non-maximum suppression trace along the edge in the edge direction and suppress any pixel value that is not considered to be an edge to zero. Phase 4 - Hysteresis - continuing the tracking until the Threshold falls behind the lower second threshold

LYU 0602 Automatic PhotoHunt Generation A5 Gaussian Filter Characteristics 1.Separable (when it is in 2 dimension0) (x,y) = (x) (y) 2.Normalized 3.Symmetric 4.Equal contribution Each pixel should have equal contribution to the father pixel in the upper level.

LYU 0602 Automatic PhotoHunt Generation