1. #? rahul swaminathan (T-Labs) & professor patrick baudisch hci2 hasso-plattner institute determining depth

4. two subproblems Matching Finding corresponding elements in the two images Reconstruction Establishing 3-D coordinates from the 2-D image correspondences found during matching

5. a little recap on reconstruction

6. camera scene lighting graphics light computer

7. camera scene lighting vision light

8. computer camera scene lighting light

9. scene the camera sees a red pixel let’s assume it correctly classifies it as “glass of red wine” screen  but, the red wine could be anywhere along this line

10. computer two cameras scene lighting light

11. triangulate the location of the actual glass

12. wine glass screens

13. two subproblems Matching Finding corresponding elements in the two images Reconstruction: done Establishing 3-D coordinates from the 2-D image correspondences found during matching

14. two subproblems Matching: harder Finding corresponding elements in the two images Reconstruction: done Establishing 3-D coordinates from the 2-D image correspondences found during matching

15. matching structured light

17. scene Could we replace one camera with a projector? two cameras lighting

18. structured light :: the process of projecting a known pattern of pixels onto a scene

20. pattern is disturbed when depth changes

21. patterns used

22. gray code 1

23. Could we achieve the same result with less images?

24. use (cos)-wave pattern instead of b/w 2

25. pattern needs processing caveat

26. Turns out to be a not too hard problem: flood-fill algorithm already provides acceptable solution

27. continues gradient result from both

28. Microsoft Kinect 3

29. Anoto pen 4

30. matching two cameras

31. computer two cameras scene lighting light

32. main approaches 1.pixel/area-based 2.feature-based

33. problems Camera-related problems - Image noise, differing gain, contrast, etc.. Viewpoint-related problems: - Perspective distortions - Occlusions - Specular reflections

34. camera positioning baseline

35. More matching heuristics Always valid: (Epipolar line) Uniqueness Minimum/maximum disparity Sometimes valid: Ordering Local continuity (smoothness)

36. Area-based matching Finding pixel-to-pixel correspondences For each pixel in the left image, search for the most similar pixel in the right image

37. Area-based matching Finding pixel-to-pixel correspondences For each pixel in the left image, search for the most similar pixel in the right image Using neighbourhood windows

38. Area-based matching Similarity measures for two windows SAD (sum of absolute differences) SSD (sum of squared differences) CC (cross-correlation) …

39. Correspondence via Correlation Rectified images LeftRight scanline SSD error disparity (Same as max-correlation / max-cosine for normalized image patch)

40. LeftDisparity Map Images courtesy of Point Grey Research Correspondence Using Correlation

41. Image Normalization Even when the cameras are identical models, there can be differences in gain and sensitivity. The cameras do not see exactly the same surfaces, so their overall light levels can differ. For these reasons and more, it is a good idea to normalize the pixels in each window:

42. matching features

43. problems

44. Scale change Rotation Occlusion Illumination ……

45. SIFT :: Scale Invariant Feature Transform; transform image data into scale-invariant coordinates relative to local features

46. result

48. combining both

50. 2 angles and one side are known  height of the triangle can be computed

51. wine glass screens

52. the underlying problem is: compute the intersection of two lines

53. commonly compute “depth” image

54. problems

55. oclusion

56. end

57. #11 professor patrick baudisch hci hasso-plattner institute title

58. :: interactive of the day

59. #11 professor patrick baudisch hci hasso-plattner institute title

61. main text is 28 pt Arial, dark gray with highlighted text is good green, and bad orange both in bold face. Include commas etc. in highlighting

62. 36pt text overlay text on 40% black 1 label

63. 1.benefit 1 2.benefit 2 benefits:

65. :: <text to define it including highlighted text only black text in deck

66. by Saturday upload storyboards for four tasks to the wiki assignment

67. E title 2x1min (this is an in-class exercise) in teams of : 1.step 1 2.step 2 Go!