SDSC The Scripps Research Institute U. Texas, Austin U.C.L.A. SIO/MIT U. California, Davis Mississippi State U. Scalable Visualization Toolkits for Bays to Brains

Motivation Support analysis, filtering, and compositing –Larger-than-core (and swap) volumetric data sets –Multi-modal and time-varying data –Multiple data sets simultaneously And... –Do efficient data movement –Execute well on parallel architectures –Integrate easily w/existing applications & toolkits –Provide interactive user interfaces for volume exploration CTCryosectionClassification

Motivating Applications Develop state-of-the-art visualization of: –Brain data 1789 TIF color images 1472 x 1152 x 1789 x 3 (bytes)= 8.5 GB –Ocean data 2160 x 960 x 30 x 4 (bytes)= 0.2 GB (MB) x 128 (time-steps)= 29.7 GB Surface height, salinity, temperature, velocity Demonstrate use beyond these fields

Toolkits Function libraries providing scalable visualization features for new and existing volumetric applications

Layered Toolkit Architecture Lowest layers provide access to larger-than-core data sets –Automatic paging of N-dimensional mesh data –Local and remote (SRB) files Higher layers provide large-data visualization –Optimized for best data movement –Multi-data set scenes –On-the-fly isocontouring during rendering Volume Scene Toolkit SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications

Mesh Toolkit Manage a paged mesh (array-like) –An N-dimensional grid of values –Spatial data & time-series –Arbitrary data content –Multiple file formats –Volume metadata –Operations like filtering, sub-sampling,... Layered atop local I/O and SRB Volume Scene Toolkit SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications

Mesh Toolkit Intelligent pre-fetch by encapsulating common access patterns –Random access (slow) –Structured access (faster) –Storage-order access (fastest) Neighborhood-aware paging –Page-in nearby data in N-dim. –Convolution filtering,... Volume Scene Toolkit SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Current center element Keep neighboring elements paged-in as well Filter window

Mesh Toolkit Status –C++ and Java implementations –Fully documented –SRB integration in progress –In active use for large data manipulation –Software release at 2001 All-Hands (with SRB) Volume Scene Toolkit SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications

Volume Scene Toolkit Define a scene containing multiple data sets –Leaf nodes are data sets, functions,... –Interior nodes composite, filter,... –Transforms align overlapping data sets Evaluate it to generate results –Client defines the scene –Server on big iron evaluates it –Layered atop Mesh Toolkit SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit Client Server

Volume Scene Toolkit Cut, Segment, Filter, and Combine data –Manage multiple data sets of arbitrary data content SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

Status –C++ prototype implementation –In active use for multiple data sets Head and brain data Tissue and cell data (Science cover) Other (Orion,...) –Software release in 2001 SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

VolRen Toolkit Data-efficient isocontouring –Isocontours produce surfaces at chosen data values Large data produces very large isocontour polygon sets –Toolkit uses a fast seed-based approach Small amount of pre-computation speeds up isocontouring on later user requests –Works from compressed data Parallel visualization of compressed streams SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit VolRen Toolkit Merged isocontouring and rendering –Isocontour extraction during volume rendering Avoids creation & storage of large polygon sets –Raycasting rays intersect volume & isosurfaces, properly mixing colors and shading Parallel out-of-core algorithm –Image-space partitioning –Master-slave structure –Low data communication cost due to data pre-processing and reorganization

VolRen Toolkit Available as a toolkit and a stand-alone system –Can integrate into applications, or –Client-server system for rendering of high-resolution videos –Client-server system for interactive exploration of time- dependent data SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit Client Server Param

VolRen Toolkit Status –C++ implementation –Additional features in development (incorporation of streaming images and geometry- progressive) –In active use for multiple data sets (structured and unstructured data) –will layer atop mesh-toolkit –Software release 1st quarter 2001 (toolkit) –remote access (client-server) release 2001 SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

Rendering Toolkit High-performance perspective volume renderer –Optimized for parallel architectures –Efficient data movement for larger-than-core rendering Available as a toolkit and a stand-alone renderer –Can integrate into applications, or –Client sends parameters –Server on big iron renders using server-side high-resolution data SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit Client Server VISTA

Rendering Toolkit Render volume meshes –Perspective ray-casting of individual volume data sets –Scenes voxelized first at arbitrary resolution SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit VISTA

Rendering Toolkit Render volume scenes –Perspective ray-casting of volume scenes –Only evaluates scene where needed –Particularly good for time-varying scenes SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit VISTA

Rendering Toolkit Status –C++ prototype implementation (based on Orion renderer) –In active use for multiple data sets –Software release mid-2001 SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit VISTA

3D GUI Toolkit Common GUI components for 3D –Generic parts for quick GUI development –3D drawing windows –3D manipulators (shape handles) –3D input device handlers –Point cloud & textured volume shapes SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

3D GUI Toolkit Status –Java partial implementation –Additional features in development –In active use in multiple 3D viewers –Software release in 2001 SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

Tools Stand-alone interactive applications and servers built using the scalable visualization toolkits

Multi-Mesh Viewer Interactive multi-mesh viewer –Built atop Mesh & 3D GUI Toolkits –Client-server access to Rendering Toolkit –Uses standard 3D graphics hardware –Single and multiple volume viewing SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit Interactive Rendered

Multi-Mesh Viewer Data-efficient visualization –Adjustable viewing resolution –Movable region-of-interest –Client-side proxy volumes for high resolution data –Client-side interaction with server-side rendering SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit InteractiveRendered

Multi-Mesh Viewer Status –Java implementation –Portable: PC, SGI, … –SRB integration in progress –In active use to interactively view meshes –Software release at 2001 All-Hands (with SRB) SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit Python Volume Viewer Hardware accelerated visualization –Built atop Mesh Toolkit + DejaVu, VLI, & Python language –Uses VolumePro rendering hardware Renders volume in real time –Interactively adjust color, lighting,... PVV

Python Volume Viewer SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit Data-efficient multi-resolution visualization –Interactive zoom between low & high resolution data PVV

Python Volume Viewer Status –Python using C++ toolkit implementation –Portable: PC, Sun, SGI –In active use to interactively visualize meshes and explore lighting, color-table, opacity and viewing parameters –Software release in 2nd quarter 2001 SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit PVV

Java Mesh Visualizer Integrated visualization tool –Built atop Mesh & 3D GUI toolkits + Shastra collaboration suite –Client-server access to data servers, renderers, and isocontouring –Web-based applet or stand-alone application SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit JVisTool

Java Mesh Visualizer Flexible visualization –Adjustable cutting planes –Isocontouring control –Integrated collaboration features –Client-side proxy volumes for high resolution data –Client-side interaction with server-side rendering & isocontouring SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit JVisTool

Java Mesh Visualizer Status –Java implementation –Portable: PC, SGI, … –In active use to visualize meshes and isocontours –Software release in 2001 SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit JVisTool

Volume Scene Editor Interactive volume scene editor –Built atop Mesh, Scene, & 3D GUI Toolkits –Client-server access to Rendering Toolkit –Diagramatic scene structure editing –Client-side interaction and editing with low-resolution proxy volumes –Server-side rendering with high-resolution volumes SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

Volume Scene Editor Data-efficient multi-mesh visualization –Efficient compositing of multiple large meshes –Multi-modal & time-varying visualization –Interactive volume scene layout –Interactive control of filters, compositing, lighting, … –Scene and viewpoint animation control SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

Volume Scene Editor Status –C++ early prototype complete –Prototype in active use –Java re-design in progress –Software release in 2001 SRB & Local I/O Isocontouring Toolkit Rendering Toolkit 3D GUI Toolkit Mesh Toolkit Applications Volume Scene Toolkit

Visualization Results New techniques to better visualize large, complex data sets - from Bays to Brains and beyond...

Brain Visualization Cleaning cryosection image data –Slices show ice and clutter around the brain –Must remove this clutter before visualization No real surface (cavities!) –Auto-segmentation uses YIQ color model, filtering, region growing, dilation, segmentation Remove exteriorDetect cavities

Brain Visualization Compressing data using wavelets –Enables multi-resolution visualization with same data size –Each step separates low and high frequency components original (256 x 256) Step 1Step 2

Brain Visualization Visualizing the cleaned brain data set –Cutting, segmenting, classifying, lighting, rendering,...

Oceanography Visualization Visualizing temperature –Temperature mapped to color –Isocontours + volume rendering

Oceanography Visualization Visualizing velocity –Vector fields atop volume rendering

Oceanography Visualization Visualizing velocity –Vector direction mapped to color –Vector magnitude mapped to intensity

SDSC The Scripps Research Institute U. Texas, Austin UCLA SIO/MIT U. California, Davis Mississippi State U. Scalable Visualization Toolkits for Bays to Brains