Introduction Autostereoscopic displays give great promise as the future of 3D technology. These displays spatially multiplex many views onto a screen, providing a more immersive experience by enabling users to look around the virtual scene from different angles.Multiview screens commonly display 8 images and future screens will certainly display even more. The most difficult task for multiview imagery is how to capture content from so many cameras. Using many cameras is complex, expensive, requires bulky equipment, and is impossible in certain applications such as medicine. Capturing imagery from two cameras, however, is much simpler and can be done cheaply with a handheld device. Thus, if we can generate many views from a stereo image pair then we can circumvent the hardware problem. The goal of view synthesis is depicted in Fig. 1. Proposed Method Our method consists of four main steps: generating an initial view, refining it, filling the holes in the disparity map, then filling holes in the color image. The inputs to the algorithm are a stereo image pair and associated disparity maps, such as the dataset shown in Fig Generating the Initial View Consider the left and right cameras to be along a normalized baseline at positions 0 and 1, respectively. Let the virtual camera be at location α, 0 < α < 1. We can map the pixels from the two views into the virtual view by horizontally shifting the pixel locations by the disparity of the pixel scaled by α or 1 − α for the left and right views, respectively. We form two candidate images, T L and T R, corresponding to the left and right images I L and I R, respectively. Letting d L and d R denote the left and right disparity maps, respectively, each of size X × Y, we have where m indexes the Y rows and n indexes the X columns of the image. When a pixel at a particular location exists in one candidate view but not the other, we have the case where a pixel is occluded in one view but not the other. These holes must be filled as described in the subsequent steps of the algorithm. The outputs of this stage are shown in Fig. 3(a) and 3(b). Let d n (C ij;r, x ij ) be the set of n smallest distances from each element in C ij;r to x ij. The color class assignment c is In the case when no boundary pixels are valid, we let c be the most common color class among valid neighborhood pixels. We use k = 5 clusters in the k-means segmentation. The final estimated image is shown in Fig. 5(a). Results The algorithm generally performs well, with an average peak signal-to-noise ratio (PSNR) of dB and structural similarity index (SSIM) of The PSNR value is only 1 dB lower than the result in view synthesis algorithm, but has the same SSIM value. Further, the run time for view synthesis algorithm is 70 seconds per image, whereas the proposed method runs about 6 times faster at 12 seconds per synthesized view. Conclusions View synthesis is a viable solution for generating content for autostereoscopic multiview displays. Furthermore, synthesizing views will be absolutely essential for certain applications and for future displays that will incorporate more views. Synthesizing views from a stereo pair also has dramatic benefits when considering data transmission of multiview imagery. Dong-Yuh Song, Hsiu-Wen Ou-Yang Department of Computer Science, Chung Cheng University, Min-Hsiung Township, Chia-yi County Literature cited EFFICIENT STEREO-TO-MULTIVIEW SYNTHESIS Ankit K. Jain Lam C. Tran Ramsin Khoshabeh Truong Q. Nguyen University of California, San Diego Department of Electrical and Computer Engineering 9500 Gilman Drive, La Jolla, CA USA Fig. 2. “Art” dataset from Middlebury Stereo Database. (a)–(b) Stereo pair. (c)–(d) Disparity maps. 2. Refinement Disparity estimates along depth discontinuities are missing from the original disparity maps in Fig. 2, or are unreliable. We tag these unreliable pixels by performing edge detection on a binary image showing the mapping of the original left and right views into the synthesized viewpoint. where ⊕ denotes image dilation, I 1 (I 2 ) is the binary map containing a 1 if the corresponding pixel was copied from the right (left) image and a 0 elsewhere, and the structural element s is given by The two operands of the union are the edge maps of the left and right mappings, and the result is dilated to increase connectivity of edges and ensure that unreliable estimates are not left unfiltered. We filter the image and disparity map outputs of Section Generating the Initial View with a 5×5 median filter at every location where M is 1 shown in fig. 4. For further information Please contact A link to an online, PDF-version of the poster is nice, too. Fig 4. Outputs from steps in Section Figure 5.. (a) Final image output of steps in Section Filling the Image. (b) Final disparity map output of steps in Section Filling the Disparity Map. EFFICIENT STEREO-TO-MULTIVIEW SYNTHESIS Fig. 1. View synthesis problem. From cameras 0 and 1, how can we estimate what a camera at position α would see? T L (m, n − αd L (m, n)) = I L (m, n) T R (m,X − n + (1 − α)d R (m,X − n)) = I R (m, X − n) M = ([(I 1 ⊕ s) \ I 1 ] ∪ [(I 2 ⊕ s) \ I 2 ]) ⊕ s 4. Filling the Image Let R i be the ith contiguous region of missing pixels and let the hole under consideration be x ij, the jth missing pixel in R i. Centered around x ij, we select an N × N neighborhood N ij. We form a histogram of B bins and find the set of pixels that are in the same bin as the estimated disparity value for x ij. The set of valid pixels V ij in N ij contains those pixels that are in the same disparity level as x ij and are not holes. We form a super neighborhood S i, which is the smallest rectangle that contains all N ij. Perform k-means segmentation with k clusters on the gray scale neighborhood. Denote the set of pixels in S i assigned to color class r as K ir, 1 ≤ r ≤ k. To determine the color class to which the hole x ij belongs, we consider the valid border pixels in each color class: We will assign a color class to x ij based on the smallest median distance of K ir to x ij. We choose the median over the same number of distances n: Fig. 3. Outputs from steps in Section 3.1.