Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz (831)

Personnel Suresh K. Lodha –PhD, CS, Rice University, 1992 –Scientific & information visualization, uncertainty quantification & visualization, multi-modal visualization, computer-aided geometric design –Uncertainty research supported by National Science Foundation & Department of Energy ASCI Program (LLNL, LANL, SNL)

Overview Uncertainty representation & computation Data/information fusion Quality-of-service issues Uncertainty visualization Uncertainty validation

Uncertainty Visualization Pipeline

Sources of Uncertainty Sensor and human limitations Noise, clutter, jamming etc. Modeling assumptions Algorithm limitations Data compression Communication errors Visualization-induced errors

Uncertainty Representation Uncertainty formalisms used by the fusion community –Probability –Dempster-Shafer evidence theory –Fuzzy sets and possibility theory Uncertainty representation in visualization research –Confidence intervals –Estimation error –Uncertainty range

Uncertainty Computation (Previous Work) Data/Information Fusion –Knowledge-based systems –Random sets (Goodman, Nguyen, Mahler) Visualization –NIST/ NCGIA `91 (Beard et al.) –BattleSpace `98 (Durbin et al.) –Visualization Software `96 (Globus, Uselton) –Scientific Visualization `96 -- (Lodha, Pang, Wittenbrink)

“Any battlefield necessarily deals with uncertainty, and it is necessary to determine ways to represent and encode the confidence level that exists for each piece of battlefield data.” – Durbin et al ’98 (NRL)

Designing Error Metrics True vs. measured/observed/anticipated Observed vs. simulated High resolution vs. low resolution Continuous vs. discrete Individual source vs. multiple sources Static vs. dynamic Time-independent vs. time-critical Error-free vs. error-prone communication

Examples of Error Metrics Local metrics -- distance metric -- curvature metric -- sampling-number or depth metric (distribution of error) Global metrics -- Topology metric

Uncertainty Metrics : Isosurfaces

Uncertainty Metrics : Fluid Flow Topology Original 332 cp 65%55%

Research Issues Representations and data structures for uncertainty measures Design and integration of error metrics Uncertainty-aware and uncertainty-reducing data processing (algorithms and models) Common consistent uncertainty representation over a distributed mobile network ?

Uncertainty Visualization How to convey uncertainty to human users? –Uncluttered display –Intuitive metaphors for mapping –Data characteristics –Multi-modality Do NOT hide processes that produce problems for the human users? –Visualize the abstraction (e.g uncertainty pipeline, graphical models)

Uncertainty Visualization Display devices /environment –screen space (monitor, PDA, workbench,..) –mobility Modality –vision –audio –haptics

Uncertainty Visualization Data types/ characteristics –scalar/vector/tensor –discrete/continuous –static/dynamic Levels of fusion –data-level (raw/abstract) –image-level (physical phenomena) –feature-level (compressed view) –decision-level (super-compressed)

Uncertainty Visualization (continued) Techniques –glyphs –deformation –transparency –texture –linking –superimposing/backgrounding –augmented reality –modality

Unc Viz: Example 3 Fluid Flow Visualization

Unc Viz: Example 5 Geometric Uncertainty

Uncertainty Visualization: Example 4

Research Issues Uncertainty mapping and metaphors for different modalities, data types and fusion tasks Display support for a variety of uncertainty metrics/formalisms Interactive display for uncertainty-source -> task analysis Integration and analysis of uncertainty for decision-making?

Uncertainty Validation Does addition of uncertainty information help human users in making decisions? Can humans integrate qualitative and quantitative (or heterogeneous information) when there is uncertainty? –Task definitions –Careful design of experiments –Usability studies –Statistical analysis

Uncertainty Validation Task definitions –primary level tasks (raw estimation) –secondary level tasks (correlation or simple spatio- temporal relationships) –higher level tasks Examples –feature existence (binary decision) –feature recognition (finite multiple choices) –target aiming (zone-centered decision within a specified space-time region)

Validation Strategies Formative vs. summative studies With or without uncertainty mapping Representative sampling of tasks, data and uncertainty mappings Constrained, interactive or free-form environment Within-subjects/between-subjects and tabular designs

Uncertainty Validation (Previous Work) Validation of user interfaces (CHI `90s) Validation of multi-modal mappings (Melara, Marks, Massaro (UCSC)) Validation of uncertainty mappings

Uncertainty Validation: Example 1 (with M. Hansen) Protein structural alignment (intuitive metaphors)

Uncertainty Validation: Example 1 (continued) Protein structural alignment -- accuracy of discrimination

Uncertainty Validation: Example 2 (GIS) Rainbow Saturated

Uncertainty Validation: Tasks(averaging, comparisons)

Uncertainty Validation: (with Wittenbrink & Pang) Vector uncertainty glyph evaluation

Research Issues Construction of user evaluation environments Conduct user evaluation studies for efficiency and accuracy Data analysis and statistical testing Feedback loop to improve performance Integrated decision tool combining uncertainty approaches in visualization and command and control?

Concluding Remarks I Provide human users with uncertainty information –Representation and computation of uncertainty –Uncertainty-aware and uncertainty-reducing algorithms and models –Uncertainty visualization –Visualization of uncertainty pipeline or hidden processes or abstract models »(continued)

Concluding Remarks II Effective and clutter-free visualization of uncertainty along with the data/information –Sensitive to data characteristics/ fusion level/ tasks/ display environments (intuitive and cognitively accurate metaphors) –Multi-modality –Usability studies

Collaboration Uncertainty representation/ fusion (UCSC, Syracuse  GTech, UCB,USC) Uncertainty visualization (UCSC, Gtech  UCB, USC) Multi-modal interaction (UCSC  USC, GTech) Other MURIS/ DoD?