Stereoscopic Imaging Systems Three-Dimensional Television F. Chen, H. Azari University of Alberta Department of Computing Science January 2008

What can you find?

Outline Human Depth Perception Factors Some Considerations in 3DTV Three Dimensional Presentation Techniques Distortions In 3D Presentation Multiview Image Sequences Coding and Signal Processing Technology How Viewing Zone Is Formed in 3DTV? Applications and Conclusion Remarks

Section Outline Human Depth Perception Factors  Why We Can See Three Dimensional?  Why We Can See Stereo TV  Which One Is Better For Us, 2D Or 3D? Some Considerations in 3DTV Three Dimensional Presentation Techniques Distortions In 3D Presentation Multiview Image Sequences Coding and Signal Processing Technology How Viewing Zone Is Formed in 3DTV? Applications and Conclusion Remarks

Why We Can See Three-Dimensional Binocular (Stereoscopic) Vision Binocular (Stereoscopic) Vision  Binocular parallax  Convergence Monocular Vision Monocular Vision  Accommodation  Motion Parallax  Occlusion  Relative size  Light and Shade  Horizon line  Aerial perspective  Linear perspective  Texture gradient

Why We Can See the Stereo TV?

The relation of the depth and the disparity length

Which One Is Better, 2D or 3D? Comparison of psychological effects between 2-D and 3-D images Sensation of Power Total picture quality

Which One Is Better, 2D or 3D? Objective evaluation of Psychological Effect The body sway of the viewer is tracked, measured and analyzed.

Which One Is Better, 2D or 3D? Eye Fatigue  Geometry, luminance and chrominance differences between the right and left images  Parallax that is very large or that rapidly changes spatially or temporally  Inconsistency between accommodation and convergence

Which One Is Better, 2D or 3D? Inconsistency between accommodation and convergence 2D3D The focal point and the convergence point fall on the point of fixation. 1. A change in the amount of parallax causes the apparent image to move away from or come closer to the screen 2. The viewer follow and converge on the movement. 3. An accommodation comes subsequently.

Which One Is Better, 2D or 3D? Inconsistency between accommodation and convergence

Section Outline Human Depth Perception Factors Some Considerations in 3DTV  Aspect ratio  Bandwidth  Cross-Talk Level  Geometry Distortions and Signal Differences  Number of viewing points  Suitable Scanning Method Three Dimensional Presentation Techniques Distortions In 3D Presentation Multiview Image Sequences Coding and Signal Processing Technology How Viewing Zone Is Formed in 3DTV? Applications and Conclusion Remarks

Some Considerations in 3DTV Aspect Ratio

Some Considerations in 3DTV Bandwidth

Some Considerations in 3DTV Cross-Talk Level

Some Considerations in 3DTV Acceptable Tolerance Geometrical Distortions and Signal Differences  Caused by insufficient positioning of two cameras and/or different focal distance of them  Acceptable tolerance: Geometry distortion: camera rotation error of 0.5deg, lens focal distance of 1% Contrast differences: signal-level differences of 1.5dB(white level) and 0.1dB(black level)

Some Considerations in 3DTV Number of View points  Flipping  60 viewing points are needed.  If the image is not very large, nine viewing points are considered adequate.

Some Considerations in 3DTV Suitable Scanning Method

Section Outline Human Depth Perception Factors Some Considerations in 3DTV Three Dimensional Presentation Techniques  Stereoscopic vs. Non-Stereoscopic Methods  Projection-Type vs. Display-Type  Parallax Barrier and Lenticular Displays  Comparison Between Different Techniques Distortions In 3D Presentation Multiview Image Sequences Coding and Signal Processing Technology How Viewing Zone Is Formed in 3DTV? Applications and Conclusion Remarks

 Non-Stereoscopic Methods Holography Holography Volumetric (3D Pixels) Volumetric (3D Pixels) Laser-Beam Scanning Laser-Beam Scanning Psychological Psychological  Stereoscopic Methods Eye-Glasses Stereoscopic Eye-Glasses Stereoscopic  Color filters  Polarizing filters  shutter glasses 3D-Image Presentation Methods Autostereoscopic Autostereoscopic  Display-Type Lenticular Parallax Barrier Integral Photography (IP) Grating Array Plates  Projection-Type Fresnel Lenz Holographic Screen Spherical Mirror

Eye-Glasses Stereoscopic  Left: polarizing glasses  Center: red-blue anaglyph  Right: PC shutter-glasses

Parallax Barrier

Lenticular

Comparison Between 3D-Methods Eye- Glasses Multi-view Holo- graphy Volumetric Laser- Beam Psycho- logical Natural Depth          Viewing Comfort     Group Viewing  Compatibility: 2D/3D     No Degrade Picture   Min Modification of Video Standard     Moderate Price      Possible  Some Cases Possible  Impossible

Characteristics of the Optical Plates Type Variable Size Method of Production Image Display Surface Lenticular Several m 2 Mechanical Optical Direct Projection Discrete Integral Photography Several 100 cm 2 Mechanical Optical DirectDiscrete Parallax Several 1000 cm 2 FilmDirectDiscrete Holographic Screen Several m 2 OpticalProjection Smooth (analog) Fresnel Lens Several m 2 MechanicalProjectionDiscrete Grating Several 100 cm 2 OpticalDirectSmooth

Section Outline Human Depth Perception Factors Some Considerations in 3DTV Three Dimensional Presentation Techniques Distortions In 3D Presentation  Distortions Source  Methods of Reducing Distortions Multiview Image Sequences Coding and Signal Processing Technology How Viewing Zone Is Formed in 3DTV? Applications and Conclusion Remarks

Distortions in Perceived Stereoscopic Images Distortion: Differences between the perceived 3D-image and the actual 3D-scene Distortion Source: Differences between photographing and viewing conditions  Geometrical: e.g. Keystone and Nonlinearity  Psychophysical: e.g. Puppet-theater and Cardboard

Distortion Solutions Using stereo camera having parallel configuration and stereo base equal to human eyes distance and preserving photographing and displaying condition the same Ortho-stereoscopic Conditions

Distortion Solutions Increasing number of views (providing motion parallax)

Section Outline Human Depth Perception Factors Some Considerations in 3DTV Three Dimensional Presentation Techniques Distortions In 3D Presentation Multiview Image Sequences  Time Multiplexing  Spatial Multiplexing Coding and Signal Processing Technology How Viewing Zone Is Formed in 3DTV? Applications and Conclusion Remarks

Multiview Image Sequences Time-Wise Arrangement (Time Multiplexing)

Multiview Image Sequences Spatial Arrangement (Spatial Multiplexing)

Section Outline Human Depth Perception Factors Some Considerations in 3DTV Three Dimensional Presentation Techniques Distortions In 3D Presentation Multiview Image Sequences Coding and Signal Processing Technology  Coding Technologies  3D Coding by MPEG  Disparity Detection in 3D Pictures  2D to 3D Conversion How Viewing Zone Is Formed in 3DTV? Applications and Conclusion Remarks

Coding and Signal Processing Technology Coding Technology  Low-level (mature) Motion compensation + waveform coding + symbol coding  Middle-level (potential) Processing of areas, layers, surfaces, depths, occlusions, and motions.  High-level (hard to achieve) Model-base coding

Coding and Signal Processing Technology 3-D coding by MPEG  Right and left channel individually coding Simple Symmetry  Utilization of right-left correlation for coding. Difficult Asymmetry (6Mbit/s, 3; 4.5, 4.5) More favorably picture quality Compatibility

Coding and Signal Processing Technology Disparity in stereo pair (3D) pictures

Coding and Signal Processing Technology Disparity detection in stereo pair

Coding and Signal Processing Technology 2D – 3D Conversion  Insufficient stereoscopic programs  For moving picture, motion vector.  For still picture, area must be detected.  Then combine the two type pictures together.

Section Outline Human Depth Perception Factors Some Considerations in 3DTV Three Dimensional Presentation Techniques Distortions In 3D Presentation Multiview Image Sequences Coding and Signal Processing Technology How Viewing Zone Is Formed in 3DTV?  Geometry of Forming Viewing Zone  Perceivable Depth  PLS, Pinhole, and Microlens Arrays  Forming Viewing Zone in Projection-Type Techniques Applications and Conclusion Remarks

Forming Viewing Zone

Obtainable Image Depth with PLS

Configurations of PLS, Pinhole and Microlens Arrays

Projection Type Auto-Stereoscopic Imaging Systems

Section Outline Human Depth Perception Factors Some Considerations in 3DTV Three Dimensional Presentation Techniques Distortions In 3D Presentation Multiview Image Sequences Coding and Signal Processing Technology How Viewing Zone Is Formed in 3DTV? Applications and Conclusion Remarks

Medicine Medicine  Distance 3D consultations and operations  3D observing diagnostics Data visualization Data visualization  CAD3/CAM4 design  Chemical and genetic molecular modeling Entertainment Entertainment  3D movie, 3D games, and 3D imaging 3D-Image Presentation Applications Cartography and meteorology Cartography and meteorology  Geographic information systems  Weather forecasting Industry Industry  3D precise modeling of engine details  Space and aircraft design simulations Architecture Architecture  Interior and exterior design  Structural analysis and building modeling

Thank you

References 1.Bahram Javidi, Fomio Okano [editors], “Three-Dimensional Television, Video, and Display Technologies”, Springer, N. Holliman, “3D display systems”, Department of Computer Science, University of Durham, Science Laboratories, South Road, Durham, DH1 3LE; Feb 2, 2005; Zhivko Yordanov, “Optimal Sub-Pixel arrangements and coding for ultra-high resolution three-dimensional OLED displays”, doctoral dissertation, Faculty of Electrical and Computer Engineering of the University Kassel, P. J. H. Seuntins, “Visual experience of 3D TV”, Eindhoven: Technische Universiteit Eindhoven, 2006, Proefschrift. 5.Anthony Vetro, Wojciech Matusik, Hanspeter Pfister, Jun Xin, “Coding approach for end-to-end 3D-TV systems”, Mitsubishi Electric Research Laboratories, Cambridge, MA.