David Luebke University of Virginia …and other cool graphics stuff

Overview l I’ll talk about a bunch of current projects: –Scanning Monticello –Immersive display –Level of detail –GPGPU –Interactive ray tracing –Graphics architectures

Scanning Monticello l Fairly new technology: scanning the world

Scanning Monticello l Want a flagship project to showcase this l Idea: scan Thomas Jefferson’s Monticello –Historic preservation –Virtual tours –Archeological and architectural research, documentation, and dissemination –Great driving problem for scanning & rendering research

Scanning Monticello—Demo

Scanning Monticello: Where To Next? l Building structured light scanner –Idea: n Shine bars of light at object n Discern its shape from their shape –Jeff Peirson is building a portable structured-light rig for scanning objects, friezework, and so on at Monticello

Scanning Monticello: Where To Next? l Capture high dynamic range color to go with scanned geometry –Show HDR, tone mapping examples –Idea: n Scan w/ laser, then take lots of pictures n Lots of pictures, different exposures, different locations (maybe even video?) n Figure out camera location very accurately n For all “correct” pixels in each image: n Project onto geometry n Store for that location and direction –Might work with Rhythm & Hues on this

Immersive Display: New Orleans Museum of Art l Jefferson’s America & Napoleon’s France –Major exhibition commemorating bicentennial of the Louisiana Purchase l Included a “Virtual Monticello” exhibit –Virtual window –Barrier stereogram l Show NOMA videos

Immersive Display: Here at UVA l We have the pieces of a cool virtual- reality immersive display system –Screens (3 silver, 2 rear-projection) –Projectors (6 dedicated, 3 others) –Head tracker –Spatial audio –Even part of the NOMA “window”! l Need to bring all the pieces together –Got a good start already

Immersive Display: Research l Design and conduct experiments with perception lab –Study the perceptual/cognitive effects of added layers of realism –Almost guaranteed to produce papers l Also fits into other research efforts –Perceptual level of detail/animation –Perceptual interactive ray tracing –Chromium

Level of Detail l A recurring theme in computer graphics: trade fidelity for performance –Reduce level of detail of distant, small, or unimportant objects 249,924 polys62,480 polys7,809 polys975 polys

Level of Detail l A recurring theme in computer graphics: trade fidelity for performance –Reduce level of detail of distant, small, or unimportant objects

Level of Detail: Motivation l Big models! –David: 56 million polygons –St. Matthew: 372 million polygons Courtesy Digital Michelangelo Project

Level of Detail Research: VDSlib l Comparatively recent technique: continuous level of detail –Adapt LOD continuously, not discrete chunks n Better fidelity/triangle n Hard to make fast on today’s hardware –VDSlib: efficient view-dependent LOD n Cool data structures & algorithms n Could make a big difference in 3D game design n Still work to be done to make fast enough

Level of Detail Research: GLOD l GLOD: –High-level library for LOD in OpenGL n Goal: make high-quality LOD easy to integrate in games and other graphics applications n Includes and interfaces with VDSlib n Cool software (interface design) issues n Free membership in the Church of Glöd –Show GLOD video

GPGPU l General-purpose computation on graphics hardware (GPUs) l Modern GPUs are fantastically capable –Extremely programmable –Full floating-point precision –Fast, so fast (20 GFLOPS sustained on GFX 5900) l We want to harness that power for scientific computing –But it’s harder than it sounds

GPGPU: CA Simulations on GPU l Promising domain: cellular automata (CA) simulations –Ex: Conway’s “Game of life” l Application: modeling cardiac tissue fibrillation –Collab with Don Jordan, MAE, and med school personnel l Possible application: simulate microvascular remodeling –Collab with Tom Skalak, BME

GPGPU: Other domains l Fluid mechanics simulation –Flow across a flapping wing –Modeling viscoelastic fluids –Joint work with Hossein Haj-Hariri, MAE –Cool math (PDEs) l Heat transfer –Simulating heat propagating across chip –Joint work with Kevin Skadron, Mircea Stan l Others…

The Big Question How should we evaluate and regulate the visual fidelity of our simplifications?

Measuring Fidelity l Fidelity of a simplification to the original model is often measured geometrically: METRO by Visual Computing Group, CNR-Pisa

Measuring Visual Fidelity l However… –The most important measure of fidelity is usually not geometric but perceptual: does the simplification look like the original? l Therefore: –We are developing a principled framework for LOD, based on perceptual measures of visual fidelity

Perceptual LOD l Idea: measure local simplification measures against a perceptual model to predict whether the user can could see the effect of simplification l Model: contrast sensitivity function

Perception 101: Contrast Sensitivity Function l Contrast grating tests produce a contrast sensitivity function –Threshold contrast vs. spatial frequency –CSF predicts the minimum detectable static stimuli

Campbell-Robson Chart by Izumi Ohzawa Your Personal CSF

Perceptual Graphics: Where To Next? l Incorporate eccentricity, velocity (attention?) l Protect copyrighted media: imperceptible “watermarking” via mesh distortion l Interactive ray tracing…

Interactive Ray Tracing l Interactive ray tracing –Once a joke, now a reality –Interesting opportunity: prioritized pixel sampling for perceptually driven rendering n Sample edges, center of gaze, etc more n Requires reconstruction of sparse samples! n Cool perceptual, signal processing issues l System: OpenRT –Would like to install on our 24-node cluster –Cool systems hacking issues

Graphics Architecture l We are exploring research topics in computer graphics architecture –Thermal-aware graphics n Joint work with Kevin Skadron –Building a simulator to explore these and other issues

My Projects: l Proposed projects: –Scanning Monticello –Immersive display –Level of detail –GPGPU –Perceptual graphics –Interactive ray tracing –Graphics architecture

Summary l Proposed projects: –Scanning Monticello n Structured light n HDR capture –Immersive display –Level of detail –GPGPU –Perceptual graphics –Interactive ray tracing –Graphics architecture

Summary l Proposed projects: –Scanning Monticello –Immersive display n System building n Perception expts –Level of detail –GPGPU –Perceptual graphics –Interactive ray tracing –Graphics architecture

Summary l Proposed projects: –Scanning Monticello –Immersive display –Level of detail n VDSlib n GLOD –GPGPU –Perceptual graphics –Interactive ray tracing –Graphics architecture

Summary l Proposed projects: –Scanning Monticello –Immersive display –Level of detail –GPGPU n CA sims n Fluid/heat sims –Perceptual graphics –Interactive ray tracing –Graphics architecture

Summary l Proposed projects: –Scanning Monticello –Immersive display –Level of detail –GPGPU –Perceptual graphics n LOD/ray tracing n Watermarking n Integrate w/ display –Interactive ray tracing –Graphics architecture

Summary l Proposed projects: –Scanning Monticello –Immersive display –Level of detail –GPGPU –Perceptual graphics –Interactive ray tracing n Perceptual heuristics n Sampling/recon –Graphics architecture

Summary l Proposed projects: –Scanning Monticello –Immersive display –Level of detail –GPGPU –Perceptual graphics –Interactive ray tracing –Graphics architecture

Summary l Proposed projects: –Scanning Monticello –Immersive display –Level of detail –GPGPU –Perceptual graphics –Interactive ray tracing –Graphics architecture l Desired student attributes: –Willing to commit hours per week –Self-starter –Good discipline & time management skills –Here over summer –Good programmer –Intro graphics (or equiv) –Hardware capable –Strong math skills –Intro perception knowledge