Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler.

Slides:



Advertisements
Similar presentations
SEMINAR ON VIRTUAL REALITY 25-Mar-17
Advertisements

Seeing 3D from 2D Images. How to make a 2D image appear as 3D! ► Output and input is typically 2D Images ► Yet we want to show a 3D world! ► How can we.
The Network of Dynamic Learning Communities C 107 F N Increasing Rigor February 5, 2011.
MTP – Stage 1 Sanobar Nishat. Outline  Peculiarities of the mobile visualization context  Different aspects of mobile visualization design  Map-based.
Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2014 Richard Fowler.
Information Visualization (Shneiderman and Plaisant, Ch. 13)
SienceSpace Virtual Realities for Learning Complex and Abstract Scientific Concepts.
CPSC 695 Future of GIS Marina L. Gavrilova. The future of GIS.
CS 5764 Information Visualization Dr. Chris North.
Lecture 4: Perception and Cognition in Immersive Virtual Environments Dr. Xiangyu WANG.
An Introduction to Software Visualization Dr. Jonathan I. Maletic Software DevelopMent Laboratory Department of Computer Science Kent State University.
Space Perception Depth Cues Tasks Shape-from-Shading.
Introduction to Virtual Environments CIS 4930/6930
2.03B Common Types and Interface Devices and Systems of Virtual Reality 2.03 Explore virtual reality.
Welcome to the world of G.V.Ram Mohan Reddy G.V.Ram Mohan Reddy.
Virtual Reality: How Much Immersion Is Enough? Angela McCarthy CP5080, SP
 Introduction  Devices  Technology – Hardware & Software  Architecture  Applications.
Visualization Technology Basic Masaki Hayashi Nov.12, 2013 Visualization of 3D CG.
Sixth Sense Technology. Already existing five senses Five basic senses – seeing, feeling, smelling, tasting and hearing.
Educator’s Guide Using Instructables With Your Students.
Information Visualization
Welcome to CGMB574 Virtual Reality Computer Graphics and Multimedia Department.
CAP4730: Computational Structures in Computer Graphics 3D Concepts.
Information Design and Visualization
Space Perception: the towards- away direction The third dimension Depth Cues Tasks Navigation Cost of Knowledge Interaction.
Chapter 5: Spatial Cognition Slide Template. FRAMES OF REFERENCE.
Introduction to Virtual Environments Slater, Sherman and Bowman readings.
© 2010 Pearson Addison-Wesley. All rights reserved. Addison Wesley is an imprint of Designing the User Interface: Strategies for Effective Human-Computer.
© 2010 Pearson Addison-Wesley. All rights reserved. Addison Wesley is an imprint of Designing the User Interface: Strategies for Effective Human-Computer.
Information Visualization Using 3D Interactive Animation Meng Tang 05/17/2001 George G. Robertson, Stuart K. Card, and Jock D. Mackinlay.
Fall 2002CS/PSY Information Visualization Picture worth 1000 words... Agenda Information Visualization overview  Definition  Principles  Examples.
VIRTUAL REALITY (VR) INTRODUCTION AND BASIC APPLICATIONS الواقع الافتراضي : مقدمة وتطبيقات Dr. Naji Shukri Alzaza Assist. Prof. of Mobile technology Dean.
Information Visualization (Shneiderman and Plaisant, Ch. 13)
VIRTUAL REALITY Sagar.Khadabadi. Introduction The very first idea of it was presented by Ivan Sutherland in 1965: “make that (virtual) world in the window.
Virtual Reality Lecture2. Some VR Systems & Applications 고려대학교 그래픽스 연구실.
GENESIS OF VIRTUAL REALITY  The term ‘Virtual reality’ (VR) was initially coined by Jaron Lanier, founder of VPL Research (1989)..
Copyright © 2005, Pearson Education, Inc. Slides from resources for: Designing the User Interface 4th Edition by Ben Shneiderman & Catherine Plaisant Slides.
Do these make any sense?. Navigation Moving the viewpoint as a cost of knowledge.
2.03 Explore virtual reality design and use.
Spatiotemporal Information Processing No.3 3 components of Virtual Reality-2 Display System Kazuhiko HAMAMOTO Dept. of Information Media Technology, School.
1 Perception and VR MONT 104S, Fall 2008 Lecture 14 Introduction to Virtual Reality.
VIRTUAL REALITY PRESENTED BY, JANSIRANI.T, NIRMALA.S, II-ECE.
1D & 2D Spaces for Representing Data Mao Lin Huang.
Digital Libraries1 David Rashty. Digital Libraries2 “A library is an arsenal of liberty” Anonymous.
Chapter 10. The Explorer System in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans On, Kyoung-Woon Biointelligence Laboratory.
© 2010 Pearson Addison-Wesley. All rights reserved. Addison Wesley is an imprint of Designing the User Interface: Strategies for Effective Human-Computer.
Thinking About Psychology: The Science of Mind and Behavior 2e Charles T. Blair-Broeker Randal M. Ernst.
Immersive Rendering. General Idea ► Head pose determines eye position  Why not track the eyes? ► Eye position determines perspective point ► Eye properties.
Foundations of Information Systems in Business. System ® System  A system is an interrelated set of business procedures used within one business unit.
 A navigational display should serve these four different classes of tasks:  Provide guidance about how to get to a destination  Facilitate planning.
Haris Ali (15) Abdul Ghafoor (01) Kashif Zafar (27)
Made By: Pallavi Chhikara
Oman College of Management and Technology Course – MM Topic 7 Production and Distribution of Multimedia Titles CS/MIS Department.
HFE 760 Virtual Environments Winter 2000 Jennie J. Gallimore
Real Time Collaboration and Sharing
Stanford hci group / cs376 research topics in human-computer interaction Information Visualization Scott Klemmer 03 November.
Information Visualization Introduction and Presentation Topics CSCI 6175 Spring 2016.
Space Perception: the towards- away direction Cost of Knowledge Depth Cues Tasks Navigation.
What is Multimedia Anyway? David Millard and Paul Lewis.
Visualization Technology Basic Masaki Hayashi Nov.10, 2015 Visualization with 3D CG.
COMP413: Computer Graphics Overview of Graphics Systems Chapter 1.
Introducing virtual REALITY
Professor John Canny Spring 2003
Information Visualization Picture worth 1000 words...
Information Design and Visualization
Virtual Reality.
AN INTRODUCTION TO COMPUTER GRAPHICS Subject: Computer Graphics Lecture No: 01 Batch: 16BS(Information Technology)
Information Visualization (Part 1)
CHAPTER 7: Information Visualization
CHAPTER 14: Information Visualization
Presentation transcript:

Information Visualization and Immersive Interfaces CSCI 6174: Open Problems in CS Fall 2013 Richard Fowler

An example Hans Rosling's 200 Countries, 200 Years, 4 Minutes

To visualize is … I see what you mean … –so, visualization can be considered not just a visual process, but a cognitive (thought) process as well …. and a very large part of human brain taken up with visual system –and that part of brain is useful beyond “simply” getting an image of the world

Visualization is … Visualize: –“To form a mental image or vision of …” –“To imagine or remember as if actually seeing …” –Firmly embedded in language, if you see what I mean (Computer-based) Visualization: –“The use of computer-supported, interactive, visual representations of data to amplify cognition” Cognition is the acquisition or use of knowledge Card, Mackinlay Shneiderman ’98 –Scientific Visualization: physical –Information Visualization: abstract

Visualization is not New Cave guys, prehistory, hunting Directions and maps Science and graphs –e.g, Boyle: p = vt … but, computer based visualization is new –Allows interactive exploration and can be embedded in the work flow –… and the systematic delineation of the design space of (especially information) visualization systems is growing nonlinearly

Visualization and Insight “Computing is about insight, not numbers” –Richard Hamming, 1969 –And a lot of people knew that already Likewise, purpose of visualization is insight, not pictures –“An information visualization is a visual user interface to information with the goal of providing insight.”, (Spence, in North) Goals of insight –Discovery –Explanation –Decision making

“... insight, not numbers” Numbers – states, %college, income : State % college degree income

“... insight, not numbers” Insights : –What state has highest income? –What is relation between education and income? –Any outliers? State % college degree income

“… insight, not numbers” Insights : –What state has highest income?, –What is relation between education and income?, –Any outliers?

Not about Useless Visual Stuff - Clutter Here, “3d” adds nothing –(at best)

Detrimental, Useless Stuff USA Today So, what information is this trying to convey?

An Example, Challenger Shuttle Presented to decision makers prior to launch –To launch or not –Temp in 30’s “Chart junk” Finding form of visual representation is important –cf. “Many Eyes”

An Example, Challenger Shuttle With right visualization, insight (pattern) is obvious –Plot o-ring damage vs. temperature

Insight, through Visualization … Some examples ….

Insight, through Visualization … What accounts for US deficit? –Economic recovery measures –TARP, Fannie, and Freddie “bailouts” –Wars in Iraq and Afghanistan –Bush-era tax cuts –Economic downturn –Other (longer standing) things

For what it’s worth …

A “Taxonomy” of Visualization Topics Card, Mackinlay, and Schneiderman Space Physical Data 1D, 2D, 3D Multiple Dimensions, >3 Trees Networks Interaction Dynamic Queries Interactive Analysis Overview + Detail Focus + Context Fisheye Views Bifocal Lens Distorted Views Alternate Geometry Data Mapping: Text Text in 1D Text in 2D Text in 3D Text in 3D + Time Higher-Level Visualization InfoSphere Workspaces Visual Objects

1D - Linear Data Software Visualization

2D - Map Data Presidential votes 2008 on Map

3D World Data – Scientific Visualization e.g., Molecule Modeling – Augmented Physical Representation

Multiple Dimensions > 3 What does visual representation for > 3 dimensions look like? “Straightforward” 1, 2, 3 dimensional representations –E.g., time and concrete Can extend to more challenging n- dimensional representations –Which is at core of visualization challenges E.g., Feiner et al., “Worlds within Worlds”

Temporal Data Baby names across years

Trees, Networks, and Graphs Connections between /among individual entities Most generally, a graph is a set edges connected by a set of vertices –G = V(e) –“Most general” data structure Graph layout and display an area of iv Trees, as data structure, occur … a lot –E.g., Cone trees

Routes of the Internet, 1/15/05 Earlier snapshot in permanent collection of NY Museum of Modern Art

Abstract, Non-physical, Network E.g., concept map –Graph of “conceptual” information What’s this one? From Tim Berners- Lee’s proposal to CERN for what is now called w w w, March 1989 Manual “graph drawing”

Text and Document Collection Structure Derivation of relationships upon which display is to be based a challenge E.g., Wise et al.

Overview Strategies Typically useful, or critical, to have “feel” for all data –Then, system allows closer inspection in “context” of all data –Overview + detail, focus + context Known from the outset of visualization –Bifocal Lens Database navigation: An Office Environment for the Professional by R. Spence and M. Apperley Shneiderman mantra –“overview first, zoom and filter, details on demand”

Focus+Context: Fisheye Views, 1 Detail + Overview –Keep focus, while remaining aware of context Fisheye views –Physical, of course –Also, distance function (based on relevance) –Given a target item (focus) –Less relevant items dropped from display –Classic cover New Yorker’s idea of the world

Focus+Context: Fisheye Views, 2 Detail + Overview –Keep focus while remaining aware of context Fisheye views –Physical, of course –Also, a distance function (based on relevance) –Given a target item (focus) –Less relevant items dropped from display –Or, are just physically smaller – distortion

Focus + Context – Spatial Distortion Selectively reduce complexity as: f(user’s viewpoint) Spatial distortion –Project network on distorted space Viewing “lens”

Focus + Context – Spatial Distortion Selectively reduce complexity as: f(user’s viewpoint) Spatial distortion –Project network on distorted space Viewing “lens” Seamless transition

Focus + Context – Hyperbolic View Again, selectively reduce complexity as f(user’s viewpoint) As with distorted lens, space is non-linear, here, hyperbolic Smooth change during interaction

3-d Hyperbolic Tree of Web Sites Using Prefuse

Visualization Pipeline: Mapping Data to Visual Form, cf. North paper Visualizations: –“adjustable mappings from data to visual form to human perceiver” Series of data transformations –Multiple chained transformations –Human adjust the transformation Entire pipeline comprises an information visualization Raw Information Visual Form Dataset Views User - Task Data Transformations Visual Mappings View Transformations F F -1 Interaction Visual Perception

Visualization Stages Data transformations: –Map raw data (idiosynchratic form) into data tables (relational descriptions including metatags) Visual Mappings: –Transform data tables into visual structures that combine spatial substrates, marks, and graphical properties View Transformations: –Create views of the Visual Structures by specifying graphical parameters such as position, scaling, and clipping Raw Information Visual Form Dataset Views User - Task Data Transformations Visual Mappings View Transformations F F -1 Interaction Visual Perception

Information Structure Visual mapping is starting point for visualization design Includes identifying underlying structure in data, and for display –Tabular structure –Spatial and temporal structure –Trees, networks, and graphs –Text and document collection structure –Combining multiple strategies Impacts how user thinks about problem - Mental model Raw Information Visual Form Dataset Views User - Task Data Transformations Visual Mappings View Transformations F F -1 Interaction Visual Perception

A Recent (2012) System

Information visualization for task of document retrieval Document collection is organized as clusters (visually represented) –User views and interacts with the cluster representations IV Interaction: –Overview –Flock-based clustering –Zoom, spatially –Filter –Text match, query –Markers –Details on demand –Document itself –Tunable

Swarm (Flock) based Clustering for Information Visualization and Retrieval Reynold’s technique for animation of flocks of birds –Computational model of group motion Particle system –Standard part of many toolkits Agent-based approach –Individual elements each apply rules for motion, as particle systems, etc. Reynolds (1987). Flocks, herds, and schools: A distributed behavioral model, Computer Graphics, 21(4),

Flock-based Movement Individual agents utilize only three rules –Agents adjust their directions of movement and velocities - steer –Form single group Movement perceptually similar to flock of birds, other social biological group –Alignment: Steer towards average direction of movement of nearby agents –Separation: Steer to avoid being too close to nearby agents –Cohesion: Steer to move toward the center position of nearby agents Rules are added for obstacles, …, edge of screen, …

Flock - based Clustering: Multiple Groups – Bird Species, Document Clusters, … Rules described so far form a single group –As, e.g., with a single species of birds Can add additional rules that consider similarity or a priori categorization among agents to form multiple groups “Multi-species flocking” –Birds of same species form groups and moved together –Agents (birds) described by feature vector, e.g., length of beak, color, size Flock-based document clustering –Vector Space Model –Similarity of documents determined by weighted number of terms in common –Essentially, a biologically (or animation) inspired “Iterative clustering technique”

System Information visualization for task of document retrieval –Overview –Flock-based clustering –Zoom, spatially –Filter –Text match, query –Markers –Details on demand –Document itself –Tunable

System’s Flock-based Clustering E.g., 500 documents –Clusters apparent Uses standard metrics of document similarity with velocity vector change based on similarity Also considers thresholds of similarity and dissimilarity –Provides tunable parameters for arranging spatial display (Cui et al., 2006) –Now,

Identifying Clusters of Interest Term Match Term match to mark documents to guide exploration –Matching documents’ color is changed –Multiple term matches

Identifying Clusters of Interest Term Match Term match to mark documents to guide exploration –Matching documents’ color is changed –Multiple term matches

Identifying Clusters of Interest Term Match Term match to mark documents to guide exploration –Matching documents’ color is changed –Multiple term matches

Identifying Clusters of Interest Term Match Term match to mark documents to guide exploration –Matching documents’ color is changed –Multiple term matches

Identifying Clusters of Interest Query Match Term match to mark documents to guide exploration –Matching documents’ color is changed –Multiple term matches Query match to mark documents

Viewing Individual Documents Zoom, then select individual element –Title is displayed

Viewing Individual Documents Select title to display individual document text –Also, can select from list retrieved in query

Interactively Adjusting Clusters In practice, many clustering techniques make use of “tuning ” System allows user to change parameters interactively –Weights for “basic” rules and other motion parameters Avoidance, cohesion, alignment Velocity minimum, maximum Limit of steering –Range for consideration –Thresholds for similarity and dissimilarity

Stereoscopic Display Perception / extraction of structure facilitated by stereoscopic display –Reveals z component of 3d arrangement –Head-tracked stereo. Structure from motion Challenge of practical stereoscopy in context of task performance Two monitor desktop

Stereoscopic Display Multi-screen display Perception / extraction of structure facilitated by stereoscopic display –Reveals z component of 3d arrangement –Head-tracked stereo. Structure from motion Challenge of practical stereoscopy in context of task performance

Immersive Interfaces, Prescence, …

Introduction The “best” interfaces, and all systems, typically find their task utility through engagement appropriate for the task –This idea is at the core of arguments for the use of direct manipulation interfaces All of the following are interrelated: –Immersion, engagement, presence, virtual reality –3D display and interaction devices In field of CS and HCI: –“spatial interfaces”, “immersive interfaces”

Examples of Immersive Interfaces Tiled display wall Head mounted display Surround screen projection Spherical projection

Immersion, “Virtual Reality”, and Virtual Environments Immersive interfaces –High sensory immersion – visual, auditory, haptic, proprioceptive “Virtual reality”, or, virtual environments –“Virtual reality is a technology that is used to generate a simulated environment in digital form… users are immersed in a totally virtual world.” –Working definition – an immersive interactive system In context of “virtual reality”, immersion = spatial immersion –Different literatures use word in different ways Note: “Immersion” (and engagement and presence) is a continuum –Text... Visual and 3d.. Stereo... HMD… “jacked in” –Cyberspace Term coined by Gibson in Neuromancer, 1984 … and in the 21 st century, the Matrix

Immersion and Virtual Reality “The mind has a strong desire to believe that the world it perceives is real” – Jaron Lanier, VR “pioneer”, … among others For example, “illusion” (perception) of depth (for spatial immersion) Stereo parallax Head motion parallax Object motion parallax Texture scale Interaction: grab and move an object Proprioceptive cues: –When you reach out and see a hand where you believe your hand should be, you accept the hand as your own Often user will accept what see asn “real” even if graphics poor Constellation of cues

optional Components for Perception of 3D for Spatial Immersion, Cutting, 1996 Perception of 3 (vs. 2) dimensions strongest element of spatial immersion –Perception of 3d from depth cues See figure Other elements –Integration important –Visual display types –Stereoscopic display –Head position sensing –Hand-position sensing –Force feedback –Sound input and output –Other sensations

Presence “The Aesthetic Impression of 3D Space” Sense of presence –Actually present in the world –Vividly 3d –Sense of being there –Holodeck … –In game literature, often called “immersion”, which is fine, but confusing for us Presence has to do with engagement, as well as visual information –E.g., one can be “in the world”, when reading –Here, one sees, or visualizes, the world

Immersive and 3D Interfaces Degree of immersion –conventional desktop screen Some 3D displays: –Data walls –Workbenches –Hemispherical display –Head-mounted displays –Arm-mounted displays –Surround screen displays - CAVE –Virtual retinal display –Autostereoscopic displays But first, … about stereoscopic display –A “compelling” 3D depth cue

3D Interfaces, Stereopsis “Discovery” of Stereopsis, 1838 Charles Wheatstone –Prolific scientist, Wheatstone bridge “… the mind perceives an object of three- dimensions by means of the two dissimilar pictures projected by it on the two retinae…”, 1838 Contributions to the Physiology of Vision.—Part the First. On some remarkable, and hitherto unobserved, Phenomena of Binocular Vision. "Philosophical Transactions" of the Royal Society of London, Vol. 128, pp. 371 – 394, What the left and right eye see (retinal image)

Getting Different Images to Each Eye Wheatstone’s Stereoscope “The stereoscope is represented by figs. 8. and 9; the former being a front view, and the latter a plan of the instrument. A A' are two plane mirrors, about four inches square, inserted in frames, …” Below, modern mirror stereoscope using computer monitors

Getting Different Images to Each Eye these days Commodity televisions –120 hz display rate –Alternately, left then right eye image –Active lcd glasses, alternately block left eye, then right eye image lcd lens is alternately transparent, then opague 60 hz left eye, 60 hz right eye Theaters –Polarized glasses

Getting Different Images to Each Eye Sutherland’s 1960’s equipment “Ultimate display”, 1965 Sword of Damocles – 1 st HMD –Actual camera-like shutters –Actual camera-like metal shutters

Immersive 3D Displays (head mounted displays) Head position is tracked, and display updated for user view –As if “in a virtual (cg) world” Head mounted display, NASA Ames, Fisher et al., mid-1980’s

Also, Virtual and Augmented Reality Augmented reality shows real world with an overlay of additional overlay Knowlton (1975) Partially-silvered mirror over keyboard Programmable labels Tactile feedback

Also, Virtual and Augmented Reality, 2 Enables users to see real world with an overlay of additional interaction –Situational awareness See through glasses Typically, add text+images to real world Very sensitive to tracking These days, “mobile apps” are appearing

Immersive 3D Displays Surround Screen Systems Essentially same multi-surface display and interaction paradigm used today as 1992 Orders of magnitude less hardware and software cost

Immersive 3D Displays Surround Screen Systems Space with walls and/or floor formed by rear projection screens –Stereoscopic –Head and position tracking So can change 3D view (projection) as user moves, or moves head Projects 3D scenes for viewer’s point of view on walls –Walls “vanish”, User perceives full 3D scene –View only correct for one viewer Visual immersion –Field of view is 100% possible

Immersive 3D Displays Surround Screen Systems Typical size: 10’ x 10’ x 10’ 2 or 3 walls are rear projection screens –To save space, mirrors often used to “fold” projection –Floor projected from above

UTPA Immersive Systems Lab ~Summer, 2013 … or so Proj. 27’ 13’ Security area 21’ 6’ TV storage ~8’ CAVE Physiological Measurement Equipment Computers Front Projection Screen Proj. Development

Current Immersive Display Projects Information visualization in immersive environments Role of stereoscopy, display size, and narrative on presence Role of personality factors in presence Gallery installation Getting things to work … –Clusters, projectors, software

Questions?