A Multiresolution Point Rendering System for Large Meshes Szymon Rusinkiewicz & Marc Levoy Stanford University Edited by Ingrid

Introduction Trend in computer graphics towards using sampled representations of real objects during renderingTrend in computer graphics towards using sampled representations of real objects during rendering Size of data meshes that can be scanned is on the order of hundreds of millions of pointsSize of data meshes that can be scanned is on the order of hundreds of millions of points Scanned data has a large number of verticies whose locations are often imprecise due to noiseScanned data has a large number of verticies whose locations are often imprecise due to noise

IntroductionIntroduction Evidenced by increased use of 3D scanners to produce three dimensional sampled models of objectsEvidenced by increased use of 3D scanners to produce three dimensional sampled models of objects This is FAR beyond what is currently displayable in real time This is FAR beyond what is currently displayable in real time Suggests an approach where individual points are relatively unimportant Suggests an approach where individual points are relatively unimportant

Introduction Examples of recent research employing this paradigmExamples of recent research employing this paradigm – Spline-fitting system by Krishnamurthy and Levoy – Range image merging system by Curless and Levoy – Octree particle rendering system by Yemez and Schmitt These algorithms do not preserve the original locations of the scanned pointsThese algorithms do not preserve the original locations of the scanned points

What is QSplat? System for representing and progressively displaying large scale meshes of scanned dataSystem for representing and progressively displaying large scale meshes of scanned data Combines a multi resolution hierarchy based on bounding spheres with a rendering system based on pointsCombines a multi resolution hierarchy based on bounding spheres with a rendering system based on points Does not preserve the connectivity of the original meshDoes not preserve the connectivity of the original mesh Intended to visualize scanned models that contain significant amounts of fine detail at scales near the scanning resolutionIntended to visualize scanned models that contain significant amounts of fine detail at scales near the scanning resolution

System Outline Preprocessing AlgorithmPreprocessing Algorithm – generates hierarchy from triangular mesh Data StructureData Structure – bounding spheres Node StructureNode Structure – sphere center, radius, normal, width of a normal cone, (optionally) a color Rendering AlgorithmRendering Algorithm – visibility culling & drawing a splat

if (begin == end) return Sphere(verticies[begin]) else midpoint = PartitionLongestAxis(verticies[begin..end]) leftstree = BuildTree(verticies[begin..midpoint]) rightstree = BuildTree(verticies[midpoint+1..end]) return BoundingSphere(leftsubtree, rightsubtree) Preprocessing Algorithm Bounding Box of vertices Recurse down each subtree, finding its bounding box

Preprocessing Algorithm Position and size of each bounding sphere is encoded relative to its parent and quantisized to 13 values to save spacePosition and size of each bounding sphere is encoded relative to its parent and quantisized to 13 values to save space Quantisized down the mesh, so errors do not propagateQuantisized down the mesh, so errors do not propagate Round up to ensure no holes introduced by quantizationRound up to ensure no holes introduced by quantization

Creating the Data Structure Start with a triangle mesh produced by aligning and integrating scans [Curless 96]Start with a triangle mesh produced by aligning and integrating scans [Curless 96]

Creating the Data Structure Place a sphere at each node, large enough to touch neighbor spheresPlace a sphere at each node, large enough to touch neighbor spheres

Creating the Data Structure Build up hierarchyBuild up hierarchy

QSplat Data Structure A single bounding sphere hierarchy used forA single bounding sphere hierarchy used for – Hierarchical frustum and backface culling – Level of detail control – Splat rendering [Westover 89]

QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits 6 bytes

QSplat Node Structure Position and radius encoded relative to parent node – Hierarchical coding vs. delta coding along a path for vertex positions Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits 2 bits 16 bits Center Offset Radius Ratio

QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits 2 bits 16 bitsUncompressed

QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits 2 bits 16 bits Delta Coding [Deering 96]

QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits 2 bits 16 bits Hierarchical Coding

QSplat Node Structure Number of children (0, 2, 3, or 4) – 2 bits Presence of grandchildren – 1 bit Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits

QSplat Node Structure Normal quantized to grid on faces of a cube Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits 52  52  6

QSplat Node Structure Each node contains bounding cone of children’s normals Hierarchical backface culling [Kumar 96] Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits

QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits Culled Not Culled Viewer

QSplat Node Structure Per-vertex color is quantized (R-G-B) Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits

QSplat Rendering Algorithm if (node not visible) Skip this branch else if (leaf node) Draw a splat else if (size on screen < threshold) Draw a splat else Traverse children Hierarchical frustum / backface culling Point rendering Adjusted to maintain desired frame rate Level of detail control

Rendering Algorithm Visibility CullingVisibility Culling – As recurse bounding spheres, cull nodes not visible – Backface culling if cone faces away, discard node & subtreeif cone faces away, discard node & subtree If cone faces viewer, mark node no cullingIf cone faces viewer, mark node no culling Frustum culling is performed by testing each sphere against planes of the view frustumFrustum culling is performed by testing each sphere against planes of the view frustum – discard nodes and subtrees that are outside the view frustum

Rendering Algorithm Determining when to recurseDetermining when to recurse – Based on projected size of screen – Node is subdivided if area of sphere when projected on viewing plane exceeds a threshold – Adjusted frame to frame to maintain user selected frame rate Type of data does not require smoothing or complex frame rate adjustmentType of data does not require smoothing or complex frame rate adjustment

Frame Rate Control Feedback-driven frame rate controlFeedback-driven frame rate control – During motion: adjust recursion threshold based on time to render previous frame – On mouse up: redraw with progressively smaller thresholds – Consequence: frame rate may vary Alternative:Alternative: – Predictive control of detail [Funkhouser 93]

PolygonsQSplat Tradeoffs of Splatting For rendering large 3D models, what are the tradeoffs of:For rendering large 3D models, what are the tradeoffs of: Good for large, flat or subtly curved regions Good for models with detail everywhere Highly-efficient rasterization with 3D graphics hardware Higher per-pixel cost, but less slowdown in absence of 3D hardware Decimation or creating LOD data structures is often expensive Fast preprocessing

Demo – St. Matthew 3D scan of 2.7 meter statue at 0.25 mm3D scan of 2.7 meter statue at 0.25 mm 102,868,637 points102,868,637 points File size: 644 MBFile size: 644 MB Preprocessing time: 1 hourPreprocessing time: 1 hour Demo on laptop (PII 366, 128 MB), no 3D graphics hardwareDemo on laptop (PII 366, 128 MB), no 3D graphics hardware

Future Work Splats as primitiveSplats as primitive – Unify rendering of meshes, volumes, point clouds – Compatible with shading after rasterization – Hybrid point/polygon systems High-level visibility / LOD frameworksHigh-level visibility / LOD frameworks – Store different kinds of data at each node: alpha, BRDF, scattering function, etc. – Potentially could be used to unify image- based-rendering (IBR) techniques

Acknowledgments Thanks to Gary King, Dave Koller, Jonathan Shade, Matt Ginzton, Kari Pulli, Lucas Pereira, James Davis, and the whole DMich gangThanks to Gary King, Dave Koller, Jonathan Shade, Matt Ginzton, Kari Pulli, Lucas Pereira, James Davis, and the whole DMich gang Digital Michelangelo Project sponsored by Stanford University, Interval Research Corporation, and the Paul Allen Foundation for the ArtsDigital Michelangelo Project sponsored by Stanford University, Interval Research Corporation, and the Paul Allen Foundation for the Arts

QSplat binaries and source codeQSplat binaries and source code Digital Michelangelo Project archive atDigital Michelangelo Project archive at QSplat Downloads