Photo VR Editor: A Panoramic and Spherical Environment Map Authoring Tool for Image-Based VR Browsers Jyh-Kuen Horng, Ming Ouhyoung Communications and Multimedia Lab. Dept. of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.

Outline  Introduction  Related works  System overview - two sub-systems Manual editor Automatic stitching method  Conclusions & Future work

Introduction  Image-based rendering becomes more and more important  Compare with geometry-based rendering constant rendering time regardless of scene complexity low computational power needed photo-realistic  How to construct a virtual environment?

Related Works  Image warping – QuickTime VR by Apple Corp.  Video clips –VideoBrush –rich frame information  Hardware sensitive –IPIX –fisheye-lens camera

System Overview

Manual Editing (1/3)  Based on real 3D graphics model Each photograph is taken as a texture of a 3D image plane All kinds of affine transformation are allowed, such as translation, rotation, scaling Pixel color is determined by multiple hit plane –ray casting –bilinear interpolation

Manual Editing (2/3)  Intensity tuning the aperture cannot be controlled - Before intensity tuning - After intensity tuning

Manual Editing (3/3)  Form panoramic image Gap closing : f’ = (360  -  g) * f / 360  f’ : adjusted focal length, f : original focal length  g : gap angle easily propagate the error blending color = Smoothing intensity discontinuity

Automatic Stitching (1/5) Camera p x 3D point p = (X, Y, Z) image coordinates x = (x, y, 1)

Automatic Stitching (2/5) 3D point p = (X, Y, Z) image coordinates x k, x kl Camera xkxk p Image k Image l xlxl

Automatic Stitching (3/5)  The relationship between p and x can be described using rotational model  The mapping between image k and l is ……(1) ……(2)

Automatic Stitching (4/5)  We wish to minimize the squared error metric g : gradient, J : Jacobian matrix  After d is solved, the matrix M is adapted by D  is the deformation matrix defined by d ……(3) ……(4)

Automatic Stitching (5/5)  To eliminate the ghosting a local search pass is introduced do 3-D search based on x-, y-, z-rotation perform an incremental update to R  : angular velocity ……(5)

Conclusions and Future Work (1/2)  Results  Performance

Conclusions and Future Work (2/2)  Some topics are under investigation An extension of the algorithm to construct spherical environment map automatically A faster and more robust method Other kind of image source (e.g. video clips)