Generation of Virtual Image from Multiple View Point Image Database Haruki Kawanaka, Nobuaki Sado and Yuji Iwahori Nagoya Institute of Technology, Japan ・

Background The soccer playing game has become popular in Japan since the World Cup Soccer of Japan and South Korea cosponsorship was held in It is desired to see the game from various view points. –setting cameras at the reverse side of the goal –at the ceiling to view down. –many cameras are set at various locations –a camera has the function of pan−tilt−zoom But, these are not free viewpoints. ・ ・

Previous method to generate a virtual image large scale environment with many camera settings and installations –It takes much cost. –the application is restricted at only that stadium. using a few cameras and a motion capture –at only the indoor space –the special wear and several markers It is difficult to use in actual games. ・・ ・ ・ ・ ・・ ・

Present approach The labels of the back number are generated as the virtual image. The pose of each player is not considered. ・・

Study Purpose generate a virtual image at another view point –from a real image –without the special environment ・・

Proposed Method The appropriate pose image of each player is determined from using multiple viewpoint image database of a player’s CG model. Each pose image is synthesized at the position to the virtual scene. The position of each player is assumed to be provided by the trajectory system.

Trajectory Recording System We have developed Trajectory Recording System. Trajectory of two players ・・・・・

Flow of Proposed Method 1.Creation of database by CG model 2.Generation of virtual image for each player from image database I.Recognize the pose of a player II.Generate the corresponding virtual image from another view point 3.Synthesis of another view point image ・

Creation of database by CG model ～ Images for Database ～ Image Database (Human model) is created by CG. –Various motions ( run, walk, shot, pass, heading, trap etc ) 280 poses –From 8 view direction (rotation with every 45°) –total 2240 images ・ ・・

Creation of database by CG model ～ Processing of Each Image ～ CG Image of human model Silhouette To eliminate of many factors such as the condition of light source, skin color, hair and uniform… –It is necessary to save the data size and the search time of the image database. Image database is created using the silhouette. –This depends on the difference of pose but does not depend on such factor of each player. ・ ・

Creation of database by CG model ～ Processing of Each Image ～ N N Normalize the image size [ ･･･ ] N×N 1-dimensional data The image size for each pose is normalize. –The rectangle region which surrounds the silhouette of each pose, is extracted. –The extracted region just touches to the square with keeping its aspect ratio. By the raster scan, one dimensional expansion of the normalized image is made. ・

Creation of database by CG model ～ Principal Component Analysis ～ Set of image data (all poses & all view) eigen values, eigen vectors Covariance matrix Compress & Projection onto eigen space If the sum of eigen values becomes over 90% effective Image data [ ･･･ ] [ ･･･ ] [ ･･･ ] ・ ・ ・ ・ ・ ・

Generation of virtual image for each player ～ Recognize the pose of a player(1) ～ Detection of player N N [ ･･･ ] by eigen vector Normarize Projection onto eigen space 1-D Each silhouette is normalized, changed to one dimensional vector and projected to a point in the eigen space. ・ ・・

Generation of virtual image for each player ～ Recognize the pose of a player(2) ～ Eigen Space Result of search minimum distance e1e1 e3e3 e2e2 CG image Real image A B When A is given, B is selected as the most similar sample. The pose of image A is recognized as B. given image ・・ ・

Generation of virtual image ～ Generate the corresponding virtual image from another view point ～ Coordinates are acquired from trajectory recording system Virtual stadium created using the OpenGL ＋ Another view image is made from result of search according to view point & view direction of virtual image. ＋ Generated virtual scene (x, y) ・ ・ ・ ・

OriginalSilhouetteSearchedSimilarity : s 90.0% 87.1% 94.0% Experiments Actual original image Virtual image from the same view direction as original The experiment of pose recognition ・ ・・・ ・・

Experiments Original image Virtual image from different view point Player’s position is fixed. Viewpoint is moved. Texture is used as soccer field. It is also possible to generate an animation of movie by connecting each frame image sequentially. ・ ・

Conclusion A new approach to generate a virtual image from another view point is proposed. –Multi-image database to apply the eigen space method for the pose recognition. –This approach is simple but generates the reasonable virtual scene. ・ ・ ・

Future Works It is difficult to discriminate the absolute pose of each player. It is also difficult t o treat the overlapped case in which two or more players cross. Investigation of more effective matching approach is required to reduce the cost of time and memory. ・