1. Visualization and Spatial Hypertext Haowei Hsieh Center for the Study of Digital Libraries Texas A&M University CPSC 436, 9/28/2006

2. Information Life Cycle Comprehension Skimming & Reading Modification Annotation & Authoring Location Searching & Browsing Located resources must be understood to be evaluated Understanding one document may require other documents or result in further information requests Added content influences further access Reading leads to annotation, note taking, and writing

3. Formality of Information People naturally process informal information People naturally process informal information By perception, association, abstraction By perception, association, abstraction Computers process formalized information Computers process formalized information Database (data tables, records, fields) Database (data tables, records, fields) Metadata (attributes, values) Metadata (attributes, values) Text Note Text & Graphics Note A Node B Web Hypertext Less formalMore formal Database Superclass Subclass Object Frames & Semantic Nets

4. Challenges Design for humans or computers? Design for humans or computers? Humans: Humans: Good at abstraction and visual perception Good at abstraction and visual perception Handle information less formally Handle information less formally Computers: Computers: Good at repetitive computation Good at repetitive computation Data needs to be formalized Data needs to be formalized

5. Research Projects Visual Knowledge Builder (VKB) Visual Knowledge Builder (VKB) An information workspace providing free-form interaction and visual expression for knowledge work with unstructured information An information workspace providing free-form interaction and visual expression for knowledge work with unstructured information VITE VITE An information workspace supporting two-way mappings to work with structured information An information workspace supporting two-way mappings to work with structured information

6. Related Research Visual languages Visual languages Specify and present relationships Specify and present relationships Express relationships in visual workspaces Express relationships in visual workspaces Explicitly via schema based visual languages Explicitly via schema based visual languages Implicitly via patterns and spatial arrangement Implicitly via patterns and spatial arrangement

7. Related Research Hypermedia/hypertext Hypermedia/hypertext Node and link hypertext Node and link hypertext Map-based hypertext Map-based hypertext Spatial hypertext Spatial hypertext Evolution of hypermedia systems Evolution of hypermedia systems NoteCardsKMSAquanetVNSTivoliHOSVIKIVITEVKB

8. Related Research Hypermedia/hypertext Hypermedia/hypertext Node and link hypertext Node and link hypertext Map-based hypertext Map-based hypertext Spatial hypertext Spatial hypertext Evolution of hypermedia systems Evolution of hypermedia systems VITENoteCards Aquanet VIKI VKB

9. Related Research Hypermedia/hypertext Hypermedia/hypertext Node and link hypertext Node and link hypertext Map-based hypertext Map-based hypertext Spatial hypertext Spatial hypertext Evolution of hypermedia systems Evolution of hypermedia systems NoteCardsKMSAquanetVNSTivoliHOSVIKIVITEVKB

10. Related Research Hypermedia/hypertext Hypermedia/hypertext Node and link hypertext Node and link hypertext Map-based hypertext Map-based hypertext Spatial hypertext Spatial hypertext Evolution of hypermedia systems Evolution of hypermedia systems NoteCardsKMSAquanetVNSTivoliHOSVIKIVITEVKB

11. Visual Knowledge Builder (VKB) Information objects: visual data representation Information objects: visual data representation Collections -- hierarchy of 2-d spaces Collections -- hierarchy of 2-d spaces Visual attributes for user interpretation Visual attributes for user interpretation Import information by drag & drop, copy/paste Import information by drag & drop, copy/paste Short Demo Short Demo

12. VKB Advanced Features Spatial parser (structure recognition) Spatial parser (structure recognition) User-defined attributes User-defined attributes Integrated search (NSDL and Google) Integrated search (NSDL and Google) Navigable history Navigable history

13. VKB Advanced Features Spatial parser (structure recognition) Spatial parser (structure recognition) User-defined attributes User-defined attributes Integrated search (NSDL and Google) Integrated search (NSDL and Google) Navigable history Navigable history Lab Administrators username: haowei group: power user

14. VKB Advanced Features Spatial parser (structure recognition) Spatial parser (structure recognition) User-defined attributes User-defined attributes Integrated search (NSDL and Google) Integrated search (NSDL and Google) Navigable history Navigable history

15. VKB Advanced Features Spatial parser (structure recognition) Spatial parser (structure recognition) User-defined attributes User-defined attributes Integrated search (NSDL and Google) Integrated search (NSDL and Google) Navigable history Navigable history

16. Example Applications Organization Organization

17. Example Applications Information gathering and collection Information gathering and collection

18. Example Applications Project management Project management

19. VKB Summary Support the information life cycle Support the information life cycle Embedded search and easy import Embedded search and easy import Visual manipulation and expression Visual manipulation and expression Support for incremental formalization Support for incremental formalization Gradually transform informal information into semi-formal or formal information Gradually transform informal information into semi-formal or formal information Software freely available at Software freely available athttp://www.csdl.tamu.edu/VKB/

20. VITE An information workspace for manipulating structured information in a semi-formal setting An information workspace for manipulating structured information in a semi-formal setting

21. Motivation Difficulties interacting with structured information Difficulties interacting with structured information Incomplete representation: deconstructive nature of formalization Incomplete representation: deconstructive nature of formalization Intermediate representation: transient states during problem solving Intermediate representation: transient states during problem solving

22. Two-way Mapping Mapping Data TableVisual Display Visualization Visual Parsing Two-way Mapping Link semantic content of the database with visual attributes of information objects

23. Semantic Attributes Visual Attributes Position, Size, Color (background and border), Text label, Border width Position, Size, Color (background and border), Text label, Border width Semantic Attributes Title, Level, Start Time, Instructor, Classroom, etc. Title, Level, Start Time, Instructor, Classroom, etc. Visual Attributes

24. Semantic Value Visual Value Mapping assignments Mapping assignments Day of Week: X Position Day of Week: X Position Start: Y Position Start: Y Position Difficulty: BG Color Difficulty: BG Color Title: Text Label Title: Text Label Start: Text Label Start: Text Label End: Text Label End: Text Label Value assignment e.g. Difficulty Value assignment e.g. Difficulty Level 1:Green Level 1:Green Level 2:Yellow Level 2:Yellow Level 3:Red Level 3:Red Mathematics Start: 1000 End: 1100

25. VITE Architecture Visual Workspace Mapping Engine Data Storage Mapping Designer Miscellaneous Interface Mapping Assignment Interface Mapping Assistant Mapping Profile User Inputs Visual Display Visual Parser Visual Renderer

26. VITE Interface Workspace Workspace Mapping Designer Mapping Designer Mapping Assignment Mapping Assignment Continuous/Discrete settings Continuous/Discrete settings Miscellaneous Interfaces Miscellaneous Interfaces

27. Mapping Designer Visual Attributes Selected Semantic Attribute Available Semantic Attribute

28. Mapping Assignment 1. Continuous value 2. Continuous color 3. Discrete value 4. Discrete color

29. Mapping Assignment 1/4 Estimated Range for Selected Visual Attribute Selected Visual Attribute : Y Size Select Attribute : MarketValue Available Value Range from Scanning the Database Select Mapping Style: Continuous Mapping Preview

30. Mapping Assignment 2/4 Default Color Range for Selected Visual Attribute Selected Visual Attribute : Background Color Select Attribute : MarketValue Available Value Range from Scanning the Database Select Mapping Style: Continuous Mapping Preview

31. Mapping Assignment 3/4 Default Values for Selected Visual Attribute Selected Visual Attribute : X Position Select Attribute : POS Available Value from Scanning the Database Select Mapping Style: Discrete

32. Mapping Assignment 4/4 Default Values for Selected Visual Attribute Selected Visual Attribute : Background Color Select Attribute : POS Available Values from Scanning the Database Select Mapping Style: Discrete

33. VITE Demo Class scheduling example used in the tutorial Class scheduling example used in the tutorial

34. User Study Investigate Investigate How people solve problems using visual workspaces How people solve problems using visual workspaces How people use two-way mappings How people use two-way mappings Observe Observe General use and selection of visual attributes General use and selection of visual attributes Identify Identify Problems when dealing with two-way mapping Problems when dealing with two-way mapping Desired enhancements through feedback Desired enhancements through feedback

35. Tasks Task 1: Flight approach control Task 1: Flight approach control User acts as an air traffic controller User acts as an air traffic controller schedule airplanes to avoid conflict schedule airplanes to avoid conflict Task 2: Fantasy basketball Task 2: Fantasy basketball User acts as a team manager of a fantasy basketball team User acts as a team manager of a fantasy basketball team Select 5 players within a limited budget Select 5 players within a limited budget

36. Task 1: Data Attributes Arrival: Arrival time Runway: Pre-assigned runway Flight: Flight Number Category:Airliner, Regional passenger plane, or Cargo plane Airline: Airline who operates this flight Airplane: Type of the aircraft (Make/Model) AP_Speed: Approach speed RW_Length: Required runway length. Capacity: Maximum number of passenger Passengers: Number of passengers on board Departure: Departure time Engine_Type: Type of engine of the aircraft, i.e., Jet, Turbo, etc. No_Engines: Number of engines Gate: Preplanned gate assignment Status: Flight status

37. Task 1: Example Mapping

38. Task 1: Example Space

39. Task 2: Data Attributes Market Value : Current salary to recruit the player POS : Position the player plays Decision : Decision to choose, reject, or still consider the player. Athlete: The name of the player Team : Where the team comes from 3PT : 3 point shots made REB : Rebounds Min : Minimum field goal AST : Assists BLK : Blocked Shots STL : Steals TO :Turnovers FTM-A : Free Throws Made-Attempted FT% : Free Throw Percentage FGM-A : Field Goals Made-Attempted FG% : Field Goal Percentage Prev : Salary of the player's previous contract Change : Change of the player's salary between current market value and previous salary.

40. Task 2: Example Space

41. Task 2: Visual Problem Solving

42. Task 2: Creating Reference

43. Task 2: Visual Sorting Object height is mapped to a critical attribute. The user need to find a proper object by comparing their height. Object height is mapped to a critical attribute. The user need to find a proper object by comparing their height. Step 1: select objects to be sorted Step 1: select objects to be sorted

44. Task 2: Visual Sorting Object height is mapped to a critical value. The user need to find a proper object by comparing their height. Object height is mapped to a critical value. The user need to find a proper object by comparing their height. Step 2: make the selection into a pile by Stack function Step 2: make the selection into a pile by Stack function

45. Task 2: Visual Sorting Object height is mapped to a critical value. The user need to find a proper object by comparing their height. Object height is mapped to a critical value. The user need to find a proper object by comparing their height. Step 3: Align the selection at top with Align Y - Top function. Step 3: Align the selection at top with Align Y - Top function. A list of objects handy for comparing their height and sorting A list of objects handy for comparing their height and sorting

46. Task 2: Comparing

47. Task 2: Result

48. Task 2: Similar Strategy

49. Observations Users’ preference of visual properties Users’ preference of visual properties Position > (Size, Color) Position > (Size, Color) Text label is inevitable Text label is inevitable Use and selection of visual properties: Use and selection of visual properties: Unmapped visual properties: temporary memory Unmapped visual properties: temporary memory Text Label: for textural description and feedback Text Label: for textural description and feedback Position: to hold status, sort, distribute, or categorize Position: to hold status, sort, distribute, or categorize Color: to show status, categorization, or reveal value distribution Color: to show status, categorization, or reveal value distribution Size: to compare value (numeric) or show categorization (discrete) Size: to compare value (numeric) or show categorization (discrete) Border Width: rarely used, because too few values available, and it effect the usefulness of border color Border Width: rarely used, because too few values available, and it effect the usefulness of border color

50. VITE Summary Two-way mapping enables semi-formal manipulation of formalized information Two-way mapping enables semi-formal manipulation of formalized information VITE supports users in developing advanced problem solving strategies VITE supports users in developing advanced problem solving strategies VITE provides users new ways of interacting with structured information VITE provides users new ways of interacting with structured information Users identified additional features that would support this approach Users identified additional features that would support this approach

51. Conclusion Information workspaces support all phases of the information life cycle Information workspaces support all phases of the information life cycle Incremental formalization facilitate the transformation of informal information into more formal knowledge representations (VKB) Incremental formalization facilitate the transformation of informal information into more formal knowledge representations (VKB) Two-way mappings enable interaction with formalized information in semi-formal environments (VITE) Two-way mappings enable interaction with formalized information in semi-formal environments (VITE)