A PROTOTYPE MULTI-VIEWER 3D TV DISPLAY Phil Surman, Ian Sexton, Richard Bates, Wing Kai Lee IMAGING AND DISPLAYS RESEARCH GROUP DE MONTFORT UNIVERSITY, LEICESTER, UK

3D TELEVISION REQUIREMENTS No Glasses (autostereoscopic) Must support multiple viewers Large viewing area Compact housing size Utilise readily-available technology Low(ish cost)

3D DISPLAY TAXONOMY Autostereoscopic Holographic Multiple Image Volumetric Real Image Holoform Multi- view Binocular Virtual Fixed Viewing Zones Head Tracking     3D DISPLAY TAXONOMY

HOLOGRAPHIC A holographic display is one where the image is produced by wavefront reconstruction The ideal stereoscopic display would produce images in real time that exhibit all the characteristics of the original scene. This would require the reconstructed wavefront to be identical and could only be achieved using holographic techniques. The difficulties of this approach are the huge amounts of computation necessary to calculate the fringe pattern, and the high resolution of the display, which has to be of the order of a wavelength of light (around 0.5 micron).

HOLOGRAPHY QinetiQ MIT OASLM EASLM Output lens Horizontal Vertical scanner Vertical scanner AOM Vertical diffuser Imaging lens QinetiQ EASLM OASLM MIT

HOLOGRAPHY Large complex hardware for small image volume High computational overhead Naturally-lit scenes difficult Unlikely for next generation TV Maybe head tracking could be used

Volumetric A volumetric display is one where the image is produced within a volume of space, and the space may be either real or virtual. Virtual image Real Image Swept volume Static volume

VOLUMETRIC Virtual Image Swept Volume Static Volume

VOLUMETRIC: PROS AND CONS Motion parallax No accommocation / convergence rivalry Image transparency. Difficult capture for video Non-Lambertian distribution difficult Swept volume not suitable for TV as this needs ‘window’ presentation

MULTIPLE IMAGE DISPLAYS In multiple image displays, two or more images are seen across the width of the viewing field. HOLOFORM: Large number of views give smooth motion parallax and hence hologram-like appearance. MULTI-VIEW: Series of discrete views presented across viewing field – these give motion parallax over limited region. BINOCULAR: Two views only presented. These may occupy fixed positions or follow viewers’ eye positions using head tracking.

HOLOFORM Holoform displays present continuous motion parallax across the viewing field QinetiQ Motion parallax Large amounts of information must be displayed Large image capture camera Cambridge Holografika

Multi-view Displays In multi-view displays, a series of discrete views are presented across the viewing field. VIEWING ZONES PARALLAX BARRIER LENTICULAR SCREEN

Philips Multiview Display 2 4 6 1 3 5 7 1 3 5 7 2 4 6 2 4 6 1 3 5 7 1 3 5 7 2 4 6 X Y A display that is so real you can almost touch the objects as they come out of the screen in 3D has been a dream for many years. But no longer claims Philips technology which is combining LCD manufacture, optical screen design and image processing software to deliver second generation 3D consumer technology.

Multi-view – Pros and Cons Simple construction Philips is 3D/2D switchable Viewing area rather limited for TV use Reduced resolution – but only factor of 3 in each direction for Philips display and factor of 2 for Sanyo 4-view.

BINOCULAR DISPLAYS Binocular, or two-image, displays may be one of three basic types: SINGLE VIEWER, FIXED VIEWING ZONES: Allows only small viewer head movement - < 65mm laterally. SINGLE VIEWER, HEADTRACKED: Enables greater freedom of head movement MULTI-VIEWER, HEAD TRACKED: The same pair of images are presented to every viewer and large freedom of movement enabled.

Sharp 2D/3D Parallax Barrier Display SeeReal Prism Mask Display Fixed Viewing Zones Sharp 2D/3D Parallax Barrier Display LEFT RIGHT Lenticular SeeReal Prism Mask Display RealityVision HOE Display

SeeReal Head Tracked Display Binocular: Single Viewer, Head Tracked PRISM MASK: SeeReal have produced a head-tracked version. HOE: A head-tracked RealityVision display is probably being developed by Samsung, but no definite information is available about this. LENTICULAR (i) : Heinrich-Hertz- Institut have produced display that enables lateral head movement. LENTICULAR (ii) : Heinrich-Hertz- Institut display developed to also allow for Z-direction. SeeReal Head Tracked Display

Multi-user, Head Tracked Binocular: Multi-user, Head Tracked Single user methods cannot be developed into multi-user displays. STEREO IMAGE PAIR ON ONE LCD SCREEN EXIT PUPILS FORMED IN VIEWING FIELD EXIT PUPIL PAIR FOR EACH VIEWER PUPILS FOLLOW VIEWERS EYES BY HEAD TRACKING

FIRST PROTOTYPE TWO-YEAR €6M PROJECT LED BY PHILIPS DMU CARRIED OUT MULTI-USER DISPLAY WORK ATTEST FINISHED IN MARCH 2004 PROOF-OF-PRINCIPLE PROTOTYPE DEVELOPED UNDER ATTEST

Exit Pupils MULTIPLE EXIT PUPILS EXIT PUPIL PAIR TOP VIEWS A SCREEN B VIEWER EXIT PUPIL PAIR L R TOP VIEWS

STEERING ARRAY REPLACES CONVENTIONAL BACKLIGHT ARRAY EFFECTIVELY SERIES OF LENSES AND LIGHT SOURCES SPACING DETERMINES DISTANCE PROVIDES 2-DIMENSIONAL CONTROL Exit pupil Illumination sources Steering array lenses Exit pupil Illumination sources Steering array lenses

STEERING ARRAY ELEMENT To viewer Aperture Driver board LED array Coaxial optical element has no off-axis aberrations. Light contained within element by total internal reflection.

IMAGE MULTIPLEXING LCDs TOO SLOW FOR TEMPORAL MUX LEFT AND RIGHT IMAGES ON ALTERNATE LINES HIGH RESOLUTION LCD (1200 X 1600) MUX SCREEN BEHIND LCD MUX screen Steering arrays To exit pupils R L Left exit pupil Right exit pupil LCD

Demonstrator Array

VIEWER B VIEWER A DEMONSTRATOR TARGETS Viewer positions determined by Polhemus 4-target head tracker

Prototype STEERING ARRAY FOLDING MIRROR SCREEN ASSY.

FIRST PROTOTYPE RESULTS – ISSUES TO BE ADDRESSED: BRIGHTNESS BANDING CROSSTALK

BRIGHTNESS ARRAY USES LOW DENSITY 3mm LEDs (ORIGINALLY MADE FOR DEMONSTRATOR) 90 x 3mm WHITE LEDs LED DRIVERS LIGHT

BANDING CIE chromaticity diagram 0.5 Y X CIE chromaticity diagram 0.5 Y Figure 4. White LED colour variation Apertures Illuminating surfaces Refracting surfaces Light to screen (a) Array element configuration (top view) (b) Appearance of aperture images (c) Intensity variation Distance across array Relative intensity

RELATIVE INTENSITY (%) LCD DIFFRACTION DISTANCE (mm) RELATIVE INTENSITY (%) 3 COMPONENTS: 270 µM PIXEL PITCH 90 µM SUB-PIXEL PITCH 15 µM MICROSTRUCTURE POINT SPREAD FUNCTION NEC LCD SUB-PIXEL MICROSTRUCTURE

FIRST PROTOTYPE USES 1800 x 3mm WHITE LEDs PERFORMANCE RELATIVELY POOR, BUT SUFFICIENT FOR PROOF OF PRINCIPLE EXIT PUPILS MOVE IN ~ 30 mm INCREMENTS EXPERIENCE GAINED USED FOR SECOND PROTOTYPE

16-element LED Array Module SECOND PROTOTYPE 16-element LED Array Module WHITE LED & LENS ARRAY DRIVER CHIP HEAT SINK LIGHT MICROLENS ARRAY CURRENTLY UNDER CONSTRUCTION 5120 WHITE SURFACE-MOUNT LEDs I6-ELEMENT LED ARRAYS WITH LENSING EXIT PUPILS MOVE IN ~ 10 mm INCREMENTS GLASS OPTICAL ELEMENTS – LESS SCATTER ANTICIPATE IMAGE WILL STILL BE DIM CROSSTALK REDUCED BY: OPERATING LCD IN PORTRAIT ORIENTATION USING MORE SUITABLE LCD DRIVERS SCATTERING REDUCED AT APERTURE AND LENS SURFACE

FUTURE RESEARCH FOLDING WILL REDUCE SIZE TO CURRENT LARGER REAR PROJECTED SETS WON’T BE SIZE OF SLIMMER REAR PROJECTED SETS AS FACETED COMPONENTS CAN’T BE USED DIFFICULT CONSTRUCTION: SURFACE-SILVERED HIGH ACCURACY VISIBILITY OF CORNERS CONSUMERS WILL DEMAND HANG-ON-WALL – FOLDING NOT SUFFICIENT DIFFERENT CONFIGURATION NEEDED LEDs MAY NOT MOST SUITABLE SOURCE: Brightness variation Colour variation Insufficient light output Large number of units COULD USE ARRAY OF BLUE JUNCTIONS WITH COMMON PHOSPHOR

HANG-ON-WALL SEMI-COAXIAL ARRAY ARRAY ELEMENT Bottom layer Top layer Apertures Refracting surfaces Light to LCD Illumination Plane TOP VIEW SEMI-COAXIAL ARRAY FLAT ILLUMINATION PLANE ACYLINDRICAL LENS SURFACE LARGE NUMBER OF INEXPENSIVE MOULDED ELEMENTS ARRAY ELEMENT

HANG-ON-WALL CONFIGURATION SCREEN ASSY. STREERING ARRAYS MIRRORS ILLUMINATION PLANES SLMs CAN BE USED (TRIED MONOCHROME LCD BUT TOO DIM) POSSIBLY USE SLM IN FOURIER TRANSFORM PLANE OF OPTICS FOR GREATER EFFICIENCY LIGHT COULD BE PIPED OR PROJECTED EVERY ILLUMINATION PLANE HAS SAME INFORMATION VIEWERS

Temporal Multiplexing TEMPORAL MUX – (IF FAST LCD NOT AVAILABLE) LEFT RIGHT STATIC MUX SCREEN LCD Static Multiplexing REAL ARRAY VIRTUAL ARRAY TEMPORAL MUX SCREEN LCD Temporal Multiplexing

2-image Head-tracked Stereo: Advantages Minimum amount of information displayed. Smallest extra bandwidth required for transmission ~ 10 - 15% (exploits redundancy in stereo pair). Simplest image capture – could be single camera pair (but might be better to have an array to enable processing).

FOCUS / ACCOMMODATION RIVALRY 2-image Stereo: Limitations A B NO MOTION PARALLAX IMAGE GEOMETRY DISTORTIONS FALSE ROTATION EYES FOCUS ON PLANE OF SCREEN EYES CONVERGE ON ‘OBJECT’ L R FOCUS / ACCOMMODATION RIVALRY

DMU’S APPROACH AIMED AT TV MARKET: PRESENT STEREO PAIR ONLY: i.e. SEVERAL VIEWERS OVER ROOM-SIZED AREA NOT SINGLE-VIEWER OR THEATRE PRESENT STEREO PAIR ONLY: NO MOTION PARALLAX BUT - LEAST AMOUNT OF INFORMATION DISPLAYED IMAGES PLACED IN VIEWING FIELD ONLY AT EYE LOCATIONS SIMPLEST CAPTURE AND TRANSMISSION HOWEVER, APPROACHES OTHER THAN TWO-IMAGE HEAD TRACKED DISPLAYS MIGHT BE APPROPRIATE, FOR EXAMPLE: MULTI-VIEW, AS CAN BE VERY SIMPLE TO IMPLEMENT HOLOFORM, WHERE REDUNDANCY IN IMAGE IS EXPLOITED VOLUMETRIC WHERE IMAGE IS OPAQUE THESE TECHNIQUES WILL BE EXPLORED WITHIN THE 3D TV NETWORK OF EXCELLENCE

3D TV NETWORK OF EXCELLENCE EU FUNDED CONSORTIUM IN FRAMEWORK 6 OF IST PROGRAMME 4-YEAR PROJECT STARTED IN SEPTEMBER 2004 150 RESEARCHERS FROM 19 ORGANISATIONS LED BY BILKENT UNIVERSITY HAS STRONG ACADEMIC BIAS WORK IS COVERED WITHIN 5 TECHNICAL COMMITTEES: TC1: 3D SCENE CAPTURE AND SCENE REPRESENTATION TC2: 3D TV CODING AND OTHER GENERIC ISSUES TC3: TRANSMISSION TC4: SIGNAL PROCESSING ISSUES IN 3D TV TC5: 3D TV DISPLAY TECHNIQUES

3D TELEVISION SPECIFIC SUPPORT ACTION (TESSA) ‘Specific support actions are intended to support the implementation of FP6, and may also be used to help prepare for future Community research policy activities.’ WILL COVER ALL ASPECTS OF 3D (NOT JUST TV) ROADMAPPING WITH QUESTIONNAIRES AND DELPHI ANALYSIS CONTACT BETWEEN NETWORKS COMPLEMENT ADRIA DISPLAYS NETWORK AND NoE WILL HAVE INFLUENTIAL STEERING GROUP – LOT OF INTEREST CURRENTLY UNDER EVALUATION - RESUBMIT SEPTEMBER IF UNSUCCESSFUL DMU 3D CONSORTIUM (INTERNATIONAL) SID RUSSIA & BELARUS CHAPTERS ASIA PACIFIC TECHNOLOGY NETWORK PHOTONICS CLUSTER (UK) ADRIA DISPLAYS SID UK CHAPTER LE CLUB VISU (FRANCE) 3D TV NETWORK OF EXCELLENCE DMU AT HUB OF 3D NETWORKING

CONCLUSIONS THE INTENTION IS FOR 3D TV TO COME TO MARKET WITHIN THE NEXT TEN YEARS. TIMING IS RIGHT AS LCD AND OTHER ENABLING TECHNOLOGIES ARE RAPIDLY EVOLVING. TWO-IMAGE HEAD TRACKING PARTICULARLY SUITED FOR 3D TV, BUT OTHER METHODS TO BE CONSIDERED ALSO.