1. Multimedia Specification Design and Production 2012 / Semester 1 / week 6 Lecturer: Dr. Nikos Gazepidis gazepidis@ist.edu.gr

2. 2 Introduction  Cameras and projectors have been intensively used in many different multimedia applications.  It is important to understand techniques to manipulate these devices.  We will study basic methods about: calibration, image warping and blending, and other issues in building a camera/projector system. Cameras & Projectors

3. 3 Example Application: Projector Geometry Distortion Compensation Cameras & Projectors Automatically correcting projector geometry distortion using a camera and projector system

4. 4 Camera / Projector Geometry Model Cameras & Projectors Camera and projector can be modeled as a pinhole imaging system Object Point Optical Center Image Plane Image point Focal length Optical axis

5. 5 Camera / Projector Geometry Model Cameras & Projectors Camera and projector can be modeled as a pinhole imaging system (X’,Y’,Z’) x = (f X’/Z’)/dx + O x = f x X’/Z’ + O x y = (f Y’/Z’)/dy + O y = f y Y’/Z’ + O y in camera’s frame is dx and dy are width and height of an image pixel f (O x, O y ) 3D point (X, Y, Z) x y

6. 6 The Projector Model Cameras & Projectors  Projector shares similar model with a camera.  The only difference is that a projector projects a flat object that emits lights based on a computer image onto an image plane, the screen  The image from the computer and the one projected on the screen are related by a homography

7. 7 Projection onto Arbitrary Surfaces Cameras & Projectors  We have learned how to deal with projector distortion using a planar screen  We can extend the method into other types of surfaces, such as cylinder or spheres

8. 8 Titled Large Screen Display Cameras & Projectors  LCD or CRT are still not easy to be made as large as a wall.  Titling multiple projector images into a large screen display is flexible and relatively cheap.  The shortcoming is we need to align the images from different projectors in both geometry and color.  Manually adjusting the projectors is a tedious task.  Camera project system can be used to solve the problem.

9. 9 Titled Images Cameras & Projectors Projector images The camera view Calibration Patterns (Projector to Camera Homography can be computed based on these patterns) 1 2 3 4 5 6 7 8 9 10 11 12

10. 10 Large Screen Projection Cameras & Projectors Projector images 1 2 3 4 5 6 7 8 9 10 11 12 The big image p camera = Hq projector p q 3

11. 11 Edge and Color Alignment Cameras & Projectors  Colors of different projectors are usually different. A color calibration and adjustment procedure is needed.  Blending 2 3 Color(p) =  Color(2,p) + (1-  )Color(3,p) p  is determined by the dominance of 2 or 3.

12. 12 Image Pyramids (Gaussian Pyramid) Cameras & Projectors A Gaussian pyramid is a technique used in image processing, especially in texture synthesis. The technique involves creating a series of images which are weighted down using a Gaussian average (Gaussian blur) and scaled down

13. 13 Image Pyramids (Laplacian Pyramid) Cameras & Projectors Uses the concept Pyramid (image processing) and is very similar to Gaussian Pyramid with the alteration that it uses a Laplacian transform instead of a Gaussian.

14. 14 Multi-band Blending Cameras & Projectors

15. 15 System Issues of Large Screen Display Cameras & Projectors  Large screen projection needs multiple projectors working simultaneously. 1. The first structure uses share memory system such as SGI Oynx2, that employs a shared-memory model. A single program can have different threads writing OpenGL primitives into different pipes while reading from a single shared database and synchronizing display update over shared flags. 2. The second structure is PC cluster, in which each PC handles one projector. This framework is much cheaper but the synchronization is a challenge problem.

16. 16 Applications Cameras & Projectors  Visualization and Collaboration IEEE Computer Graphics and Applications, 2000

17. 17 Input Methods for Large Screens Cameras & Projectors  It presents challenges for traditional input methods.  Possible HCI methods include: 1. Pointing devices, including 3D tracking, passive optical (video) tracking, ultrasonic tracking, mice, and tablet interfaces; 2. User tracking, for point-of-view rendering or for gaze directed interaction, via optical tracking or electromagnetic tracking; 3. Handheld devices, providing control interfaces that can be out of band from the display; 4. Voice commands with audio feedback; 5. And haptics interfaces.

18. 18 Multiple Camera System Cameras & Projectors  Multiple camera system can capture video from different locations simultaneously.  Stereo system has been widely used for inferring the “depth” of objects.

19. 19 Multiple Camera System (cont.) Cameras & Projectors

20. 20 Application of Multi-camera Imaging Cameras & Projectors

21. 21 Cameras in the Matrix Cameras & Projectors