1 Web3D Using Expressive Rendering for Remote Visualization of Large City Models Wednesday, April 19 Jean-Charles Quillet Gwenola Thomas Xavier Granier Pascal Guitton Jean-Eudes Marvies

2 Web3D Introduction Remote visualization is concerned with improving realistic rendering Complex 3D models Complex lighting system Progressive texture transmission For trip or assisted navigation The solutions mostly rely on 2D information Available on PDA and mobile devices combined with GPS Powerful navigation tool Remote visualization city models on mobile devices

3 Web3D Introduction Remote visualization context The server disposes of the entire geometry of the virtual city It sends a subset of the city model to the client on demand Virtual cities Simplified geometry Textured by photographs of real building facades Issues Limited network bandwidth Not adapted to the small size of the display

4 Web3D Our approach Expressive or non-photorealistic rendering (NPR) with feature lines Making more legible image More rapid transmission for data We present a pipeline able To extract the feature lines from photographs of building facades To transmit and render them on clients

5 Web3D Previous work NPR for remote rendering [Hekmatza et al. 2002] and [Diepstraten et al. 2004] Lines are computed for each new viewpoint Adapted to smooth 3D models For cities, feature lines are related to the façades 3D model ServerClient Viewpoint Extracted lines 2D Line-based rendering

6 Web3D System overview Buildings extrusion Extraction of feature lines Spatial subdivision Visibility computation Model streaming Line rendering City footprints + Textures NPR database Optimized database NPR modeling Database optimization Client-server visualization Offline Online

7 Web3D Scene subdivision Goal: produce a 3D database ready to be streamed using visibility information VRML97 extension for visibility streaming [Marvie et al. 2004] Scene subdivision in cells to produce the navigation space Constrained BSP subdivision [Marvie PhD 2003] Computation of Potentially Visible Object Set (PVOS) for each cell Compute cell-to-objects visibility relationships OpenGL shooting algorithm [Marvie PhD 2003]

8 Web3D Visibility streaming The process is initiated by the client Buffering step: –the starting cell is downloaded in a synchronous way together with its PVOS During navigation: –Prediction is made about the next visited cell –This cell and its adjacent cells are downloaded together with their PVOS in an asynchronous way

9 Web3D Feature extraction of the facades Edge detection Vectorization Rendering

10 Web3D Edge detection Classical approach Gradient approximation using a convolution kernel: Sobel [Pingle 1969], Roberts [1965], Prewitt [1970] Simple thresholding The contours are not 1 pixel wide, needed for vectorization Canny edge detector Gaussian blur Gradient computation Non-maximal suppression: insures 1 pixel wide contours Hysteresis thresholding

11 Web3D Pixel chaining 1.Contours 2.Junction and end-points detection 3.Graph initialization 4.Graph filling 5.Loop processing 6.Vectorization

12 Web3D Vectorization Vectorization of the pixel chains Minimization of the distance between the approximation and the curve [D.H. Douglas 1973], [de Figueiredo 1995] Best approximation of the angular [Dunham 1986] or the curvature [Asada and Brady 1986] Minimization of the area between the approximation and the curve [Wall and Danielsson 1984] Post processing and cleaning: simple heuristics Junction correction Noise removing Line straightening

13 Web3D Rendering of lines Lines are transmitted as IndexedLineSet Lines are rendered with a constant width of one pixel At some point the screen is saturated with lines Need to remove some lines

14 Web3D Rendering of lines Level of details Depending on the distance the façade is viewed We draw a subset of the lines Advantages Reducing the line density on screen Less geometric primitives are drawn, rendering time is improved Line selection criterion Based on line length Same lines are used between each level Visual continuity between the different levels

15 Web3D Results We compare our approach to textured rendering On data size Rendering speed Hardware: DELL Axim X50V 624 MHz ARM processor 64 MB of RAM memory Intel 2700 GPU Software based on OpenGL-ES standard Magellan framework for WinMobile (VRML/X3D based streaming & rendering) Intel SDK for hardware acceleration Software-based library developed by Hybrid Graphics

16 Web3D Results Initial data set Fixed viewpoint –Low resolution pictures (250x320): French yellow pages website –High resolution pictures (3008x2000): digital camera City walkthrough –Model and textures maps of Bordeaux [Hachet Guitton 2001] –Small texture map sizes (~64x64)

17 Web3D Fixed viewpoint Data size Vector data smaller than original pictures Drawn images smaller than original pictures KB37.55 KB KBHigh resolution 3.13 KB4.16 KB8.55 KBLow resolution Drawn images PNGVector dataJPEG

18 Web3D Fixed viewpoint Rendering speed Using low resolution images –On software-based rendering – texture ~ lines + LOD: 6 fps –Using hardware acceleration –Texture: 39.6 fps –Lines + LOD: 17.3 fps (because no display lists on GL|ES) Using high resolution images –Still practicable using lines + LOD: 3 fps –Results for lines are better using software-based rendering –no transfer on the bus for SW based Significant better performances using LOD than full line rendering

19 Web3D Moving viewpoint: the complete city model Record statistics during a city walkthrough Memory used Frame rate Rendering time Scene graph traversal Download bitrate Using different models Geometry only as reference Textured based model Model together with its full line appearance description Model that takes advantage of the presented LOD technique

20 Web3D Moving viewpoint: the complete city model Average size of downloaded files Textures: KB Lines: 7.55 KB Memory occupation 2 MB smaller for the lines than for textures

21 Web3D Moving viewpoint: the complete city model Rendering speed (SW vs HW) Textures: 2.18 fps / fps Lines: 1.17 fps / 1.25 fps Lines + LOD: 2.28 fps / 5.45 fps In software Lines + LOD ~ textures (because of their small resolution) In hardware Texture gets better result (because of bus transfers)

22 Web3D Moving viewpoint: the complete city model The rendering is divided in two steps Scene graph traversal time Rendering time Scene graph traversal time Texture: 12 ms Lines + LOD: 61 ms The scene graph is simpler for the texture-based rendering Rendering time With hardware acceleration: –Texture: 90 ms –Lines + LOD: 258 ms With software library: –Texture: 388 ms –Lines + LOD: 347 ms Improve the scene graph traversal time by implementing a dedicated node

23 Web3D Moving viewpoint: the complete city model Bitrate measurement: the rate the data are sent by the server to the moment they are available in the client scene graph. 1. Data transmission over the network 2. Data decompression 3. VRML97 parsing and conversion to fixed point 4. Node initialization Resulting bitrate for the lines: bps for the textures: bps Lines bitrate lower than map’ ones due to conversion to fixed point Might need fixed point fields types in VRML/X3D

24 Web3D Conclusion Our approach Based on expressive rendering Features lines are extracted from original texture The resulting data set is smaller than the original one Experiments On PDA The result conveys the required information The amount of data to transmit is greatly reduced We can obtain interactive rendering even using software 3D rendering

25 Web3D Future work Feature lines extraction Improve its robustness and efficiency Perform a cognitive study on scene recognition Rendering system Use up-coming vector processor based on OpenVG standard Allow integration of larger range of NPR styles

26 Web3D Thank you