Синтез изображений по изображениям. Рельефные текстуры

Сегодня на лекции Введение в Синтез Изображений по Изображениям (Image-Based Rendering)Введение в Синтез Изображений по Изображениям (Image-Based Rendering) Простейшие методы IBRПростейшие методы IBR Рельефные текстуры (relief textures)Рельефные текстуры (relief textures) Введение в Синтез Изображений по Изображениям (Image-Based Rendering)Введение в Синтез Изображений по Изображениям (Image-Based Rendering) Простейшие методы IBRПростейшие методы IBR Рельефные текстуры (relief textures)Рельефные текстуры (relief textures)

Traditional Rendering User input texture maps, survey data User input texture maps, survey data Geometry Textures Light sources Geometry Textures Light sources Images Modeling Rendering For photorealism Modeling is hardModeling is hard Rendering is slowRendering is slow For photorealism Modeling is hardModeling is hard Rendering is slowRendering is slow

Image-Based Rendering Основные идеи: Использование изображений (фотографий) в качестве исходных данныхИспользование изображений (фотографий) в качестве исходных данных Использование методов обработки изображений для визуализации.Использование методов обработки изображений для визуализации. Основные идеи: Использование изображений (фотографий) в качестве исходных данныхИспользование изображений (фотографий) в качестве исходных данных Использование методов обработки изображений для визуализации.Использование методов обработки изображений для визуализации.

Image-Based Rendering Images user input range scanners Images user input range scanners Images & depth maps Images Modeling Rendering For photorealism Fast modelingFast modeling Complexity independent renderingComplexity independent rendering For photorealism Fast modelingFast modeling Complexity independent renderingComplexity independent rendering

Traditional vs. Image-based Imagebased computer graphics has three main advantages Image based computer graphics has three main advantages Photorealism of produced imagesPhotorealism of produced images The speedThe speed Simple modelingSimple modeling Imagebased computer graphics has three main advantages Image based computer graphics has three main advantages Photorealism of produced imagesPhotorealism of produced images The speedThe speed Simple modelingSimple modeling

Simplest IBR methods. Texture Texture - is the simplest of IBR methods.

Simplest IBR methods. Sprites Texture + simple planar geometry = Sprite Sprites are taken from certain camera position (sprite camera) Texture + simple planar geometry = Sprite Sprites are taken from certain camera position (sprite camera)

Simplest IBR methods. Sprites No geometry information and... Sprites are looking good from view-points close to the sprite camera, but awful from others No geometry information and... Sprites are looking good from view-points close to the sprite camera, but awful from others

3D model rendering distortions What are we missing? The effect of 3-dimensionality on the screen is a combination of two effects: perspective distortion and parallax Sprites are capable of producing perspective distortions but they are unable to produce correct parallax effect What are we missing? The effect of 3-dimensionality on the screen is a combination of two effects: perspective distortion and parallax Sprites are capable of producing perspective distortions but they are unable to produce correct parallax effect

Sprites with Depth Combine depth and color: Color texel now is a 3D sample. Combine depth and color: Color texel now is a 3D sample.

Warping Using the samples z- values, image can be transformed (or warped), to enchance the image descriptive power (realism)

Planar pinhole camera

Orthographic camera

3D point position from a pinhole camera One image is not enough to determine location of a point in 3D.

From two cameras If we have two cameras, Camera 1 and Camera 2 with different parameters, capturing the same scene from different locations, a point can be expressed as

Warping example

Schematic view Depth map Final view Warping x’ = warp(x) Parallax, perspective projection, translation

Relief textures Most of modern 3D accelerators can ultimately fast render textured triangles => We can use this capability to speed up and simplify Image-Based Rendering Such an algorithm is called Relief Textures Most of modern 3D accelerators can ultimately fast render textured triangles => We can use this capability to speed up and simplify Image-Based Rendering Such an algorithm is called Relief Textures

Schematic view Relief textures Final view Pre-warping Parallax Warped textures Texture mapping Perspective projection, translation Software Hardware

New warping function x’ = warp(x) = g(h(x), Poly), where –g(y, Poly) is texture-mapping function and usually done in hardware –h(x) is pre-warpring function x’ = warp(x) = g(h(x), Poly), where –g(y, Poly) is texture-mapping function and usually done in hardware –h(x) is pre-warpring function

New warping function (2) Prewarping function h(x) after some optimizations looks like following: u2 = (u1+mu[d])*nu[d]; v2 = (v1+mv[d])*nv[d]; Extremely simple, isn’t it? Prewarping function h(x) after some optimizations looks like following: u2 = (u1+mu[d])*nu[d]; v2 = (v1+mv[d])*nv[d]; Extremely simple, isn’t it?

Two stage warping example

Two stage warping example (2)

Filling holes This sprite with depth was warped to the new viewpoint Look how many empty spaces on the women face and hair at the picture Let’s call them holes This sprite with depth was warped to the new viewpoint Look how many empty spaces on the women face and hair at the picture Let’s call them holes

Two classes of holes All the holes fall into two classes by its nature Resampling problemResampling problem Missing informationMissing information All the holes fall into two classes by its nature Resampling problemResampling problem Missing informationMissing information

Resampling Before warping After warping

Resampling methods Methods to fill the holes Inverse warpingInverse warping MeshingMeshing SplattingSplatting InterpolationInterpolation Methods to fill the holes Inverse warpingInverse warping MeshingMeshing SplattingSplatting InterpolationInterpolation

Splatting Draw a little cloud (splat) instead of a pixel in desired image. This cloud has to be opaque in its center becoming more and more transparent to its sides Features Relatively small computation costRelatively small computation cost Not all the holes are filledNot all the holes are filled Draw a little cloud (splat) instead of a pixel in desired image. This cloud has to be opaque in its center becoming more and more transparent to its sides Features Relatively small computation costRelatively small computation cost Not all the holes are filledNot all the holes are filled

Interpolation Use the fact that u and v are independent from each other after pre-warping (two-pass algorithm) and linearly interpolate depth and color in the intermediate and final images

Interpolation (2)

Пример 1 Пример 2