1. LYU 0602 Automatic PhotoHunt Generation Automatic PhotoHunt Generation Shum Hei Lung To Wan Chi Supervisor: Prof. Michael R. Lyu

2. LYU 0602 Automatic PhotoHunt Generation Background Objectives System Implementation Segmentation Module Elimination Module Other Module Web-based PhotoHunt game Conclusion Overview

3. LYU 0602 Automatic PhotoHunt Generation Background PhotoHunt is … –a Spot-the-difference game –Classic yet evergreen –Popular in electronic game centers all over the world However… It is limited by man power

4. LYU 0602 Automatic PhotoHunt Generation Basic techniques to edit image Elimination Color modification Cloning Transformation

5. LYU 0602 Automatic PhotoHunt Generation Objectives To implement the PhotoHunt Generation Engine –Focus on eliminating object Develop an engine to support real-time image generation for PhotoHunt game

6. LYU 0602 Automatic PhotoHunt Generation System Implementation

7. LYU 0602 Automatic PhotoHunt Generation Image Generation Engine To generate an image for PhotoHunt game –Effects that may be applied: Elimination Color Change Duplication Definition of well generated image: The effects should be “NOT OBVIOUS YET DISCOVERABLE”

8. LYU 0602 Automatic PhotoHunt Generation Image Generation Engine – The Process Segmentation Module Modification Modules Smooth Image GAME Game Engine

9. LYU 0602 Automatic PhotoHunt Generation Segmentation Module

10. LYU 0602 Automatic PhotoHunt Generation Segmentation Module To detect and extract segment from the input image Three Phases: 1.Segmentation 2.Constraint Checking Area of Input Image/10000<Area of Segment < Area of Input Image/500 3.Area of Segment Reference image building 000000000 00000000 00000000 000000000 000000000 000000000 000000000 000000000 000000000 1111

11. LYU 0602 Automatic PhotoHunt Generation Segmentation Algorithm The Segmentation algorithms we have examined: Pixel-based Algorithm Edge Detection Algorithm Region-based Algorithm

12. LYU 0602 Automatic PhotoHunt Generation Gaussian Pyramid Pyramid Segmentation The Pyramid Segmentation Algorithm Step 1 Generation of the Gaussian pyramid Step 2 Segmentation by pyramid-linking and averaging linked-pixel

13. LYU 0602 Automatic PhotoHunt Generation Step 1- Generation of the Gaussian pyramid W(-2)W(-1)W(0)W(1)W(2) 5 5 8 7 6 5 4 + g(i)Father node Level l-1 Level l

14. LYU 0602 Automatic PhotoHunt Generation Step 1- Generation of the Gaussian pyramid

15. LYU 0602 Automatic PhotoHunt Generation Gaussian Pyramid Original photo g0 Dimension= 300*300 Level 1: 150*150 Level 2: 75*75 Level 3: 36*36 Step 1 Computation of the Gaussian pyramid - Step 2 Segmentation by pyramid-linking

16. LYU 0602 Automatic PhotoHunt Generation Step 2 Segmentation by pyramid-linking and averaging linked-pixel c t (i,j,l) : local characteristics f t (i,j,l) : The potential father node s t (i,j,l) : segment property area t (i,j,l) : area of the segment

17. LYU 0602 Automatic PhotoHunt Generation 1.Set f t (i,j,l) 2.Initialize area t (i,j,0) =1 c t (i,j,l) =c 0 (i,j,0) 3. for l = 0 to L s t (i,j,L) = c t (i,j,L) for l = L downto 0 s t (i,j,l) = c t (i”,j”,l+1) Step 2 Segmentation by pyramid-linking and averaging linked-pixel

18. LYU 0602 Automatic PhotoHunt Generation 224 121512 13 17 21 17 4422 Step 2 Segmentation by pyramid-linking and averaging linked-pixel

19. LYU 0602 Automatic PhotoHunt Generation Segmentation Result

20. LYU 0602 Automatic PhotoHunt Generation Elimination Module

21. LYU 0602 Automatic PhotoHunt Generation Elimination Algorithm Direct Copy Algorithm Horizontal Gradient Algorithm Nearest Boundary Algorithm Enhanced Nearest Boundary Algorithm

22. LYU 0602 Automatic PhotoHunt Generation Direct Copy Algorithm Directly copy the upper pixel to current pixel Where S i is segmented region of the image for i = 0,1,2, ……

23. LYU 0602 Automatic PhotoHunt Generation Direct Copy Algorithm

24. LYU 0602 Automatic PhotoHunt Generation Horizontal Gradient Algorithm Determine color line by line Smoothing the changes of color from one side to another Where C(M) is the color verctor of pixel M and d XY is the distance between X and Y

25. LYU 0602 Automatic PhotoHunt Generation Horizontal Gradient Algorithm

26. LYU 0602 Automatic PhotoHunt Generation Horizontal Gradient Algorithm

27. LYU 0602 Automatic PhotoHunt Generation Horizontal Gradient Algorithm

28. LYU 0602 Automatic PhotoHunt Generation Nearest Boundary Algorithm Where d min = min(d PN, d QN, d RN, d SN ) Replace the color by the color of nearest boundary

29. LYU 0602 Automatic PhotoHunt Generation Nearest Boundary Algorithm

30. LYU 0602 Automatic PhotoHunt Generation Enhanced Nearest Boundary Algorithm Improved from Nearest Boundary Algo. Where d min = min(dPN, dQN, dRN, dSN) S(A,B) is Euclidean distance between A and B

31. LYU 0602 Automatic PhotoHunt Generation Enhanced Nearest Boundary Algorithm

32. LYU 0602 Automatic PhotoHunt Generation Enhanced Nearest Boundary Algorithm

33. LYU 0602 Automatic PhotoHunt Generation Others Modules

34. LYU 0602 Automatic PhotoHunt Generation Color Change Module Segment dominate color Operation apply RedSwap red and blue channel GreenSwap red and green channel BlueSwap green and blue channel Dominate color segment: % of Pixel of dominate color> 75% Dominate color pixel: DChannel >180 and otherChannel<50

35. LYU 0602 Automatic PhotoHunt Generation

36. Smooth Image To reduce noise and distortion To make the image more realistic Gaussian Filter (Neighbor size=3, sigma=1)

37. LYU 0602 Automatic PhotoHunt Generation Semi-Automatic PhotoHunt Generation

38. LYU 0602 Automatic PhotoHunt Generation Semi-Automatic PhotoHunt Generation

39. LYU 0602 Automatic PhotoHunt Generation Game Engine

40. LYU 0602 Automatic PhotoHunt Generation Game Engine

41. LYU 0602 Automatic PhotoHunt Generation Game Engine – Front Page

42. LYU 0602 Automatic PhotoHunt Generation Conclusion

44. Limitation Only elimination and limited color change effect supported No artificial intelligence control to the Applied effect Modified area found noise and distortion Not all user input can be segmented

45. LYU 0602 Automatic PhotoHunt Generation Looking Forward… Improve the segmentation algorithm Implement the Duplication and Transformation module Support numbers of object recognitions More Features on the Web-based game Ultimately, to achieve the complete automatic PhotoHunt generation

46. LYU 0602 Automatic PhotoHunt Generation Q & A

47. LYU 0602 Automatic PhotoHunt Generation The End Thanks for your kind attention.

48. LYU 0602 Automatic PhotoHunt Generation AB CD A1 Example of Pyramid Segmentation 212520 221511 131123 1312 2 c 0 (0)= 21 c 0 (1)= 25 c 0 (2)= 2 c 0 (3)= 0 c 0 (4)= 22 c 0 (5)= 15 c 0 (6)= 1 c 0 (7)= 1 c 0 (8)= 13 c 0 (9)= 11 c 0 (10)= 4 c 0 (11)= 5 c 0 (12)= 13 c 0 (13)= 12 c 0 (14)= 12 c 0 (15)= 2 c 0 (A)= 21.2 c 0 (B)= 2.2 c 0 (C)= 14.8 c 0 (D)=3.2 f 0 (0)= A f 0 (1)= A f 0 (2)= B f 0 (3)= B f 0 (4)= A f 0 (5)= C f 0 (6)= B f 0 (7)= B f 0 (8)= C f 0 (9)= C f 0 (10)= B f 0 (11)= D f 0 (12)= C f 0 (13)= C f 0 (14)= C f 0 (15)= D a 0 (A)= 3 a 0 (B)=5 a 0 (C)= 6 a 0 (D)= 2 c 0 (A)= 22.6 c 0 (B)=1.2 c 0 (C)= 12.6 c 0 (D)= 2.5

49. LYU 0602 Automatic PhotoHunt Generation A3 Pixel-based Segmentation - Thresholding Fig 3.2b Histogram of data generated by ThresSeg.cpp Intensity (0-2 8 ) Frequency Threshold

50. LYU 0602 Automatic PhotoHunt Generation A4 Edge Detection Segmentation- Canny Edge Detection Phase1 - Gaussian convolution S(x,y)=smoothed image Phase 2 - Edge strength and direction |M|=|H|+|V| Phase 3 - Non-maximum suppression - Non-maximum suppression trace along the edge in the edge direction and suppress any pixel value that is not considered to be an edge to zero. Phase 4 - Hysteresis - continuing the tracking until the Threshold falls behind the lower second threshold

51. LYU 0602 Automatic PhotoHunt Generation A5 Gaussian Filter Characteristics 1.Separable (when it is in 2 dimension0) (x,y) = (x) (y) 2.Normalized 3.Symmetric 4.Equal contribution Each pixel should have equal contribution to the father pixel in the upper level.

52. LYU 0602 Automatic PhotoHunt Generation