1. CS 990: Topics in Scientific Visualization Research
Dr. Mai Elshehaly
Department of Computer Science
Suez Canal University
© Mai Elshehaly

2. Class Rules & Expectations
This class simulates a research lab setting and offers a head start for life in academia.
The expectations are the same as those of the majority of research labs in Egypt and abroad.
Participation is a MUST
Verbal communication is in English
Electronic communication is timely and crucial: check your at least once a day
Deliverables.. Deliverables.. Deliverables
© Mai Elshehaly

3. Learning Objectives
By the end of this year-long course you should gain proficiency in:
Defining a problem for research
Characterizing the problem, the underlying data, and user-specified requirements
Formulate user tasks for visualization
Reflect on existing visualization techniques that tackle your identified problem and develop your own variants
Understand the difference between quantitative user studies and qualitative ways to evaluate the visualization
© Mai Elshehaly

4. Course Structure 1 weekly meeting: Weekly deliverables:
Quiz from half a book chapter
2 paper discussions:
Two team members will present the two papers (one for each member)
The remaining team members will select only one of the papers to read carefully, and will prepare 1-2 questions on the other paper
Presentations will be performed in a round robin
A tutorial on implementation issues
Weekly deliverables:
Presentations (if it is your turn)
Questions on papers
In class Quiz answers and other in class activities
An implementation OR writing assignment
© Mai Elshehaly

5. Final Deliverable A final report in the form of a paper.
© Mai Elshehaly

6. Ways to reach me
Mobile: Web page:
© Mai Elshehaly

7. The Parallel Track
There are a few students who are interested in auditing the class and collaborating on paper writing.
The parallel track is for them.
An exchange of ideas between the tracks is applauded and valued.
© Mai Elshehaly

8. Grading
Grading for this course will be gradual and continuous based on performance.
Weekly tasks are assigned a total of 7 points per week.
Each day of delay means -1 point
An assignment delayed for more than 1 week will not receive any grade
© Mai Elshehaly

9. Today’s Paper
 Design Study Methodology
© Mai Elshehaly

10. Paper Discussion Outline
Identify:
Why is the problem that the paper tackles important? (motivation)
What is the solution that the paper is presenting? (contribution)
How does the paper attempt to solve the problem? (technique)
Reflect:
Where does the paper fall short on addressing the problem (room for extension)
Why was the paper accepted
What would you do to extend on the paper
© Mai Elshehaly

11. Paper Discussion Outline
Identify:
Why is the problem that the paper tackles important? (motivation)
What is the solution that the paper is presenting? (contribution)
How does the paper attempt to solve the problem? (technique)
The above questions MUST be answered in the paper’s abstract, and elaborated on in the introduction
© Mai Elshehaly

12. WHY
WHAT
HOW
© Mai Elshehaly

13. Motivation (Problem statement)
Design studies are an important form of problem-driven research in the field of information visualization
There is not enough guidance on how to perform successful design studies
This is what we learned from Abstract
How does Introduction elaborate?
© Mai Elshehaly

14. © Mai Elshehaly

15. How did the Introduction elaborate?
More detailed description of the problem
Definitely NOT a copy from abstract
Citing references to back the claim that this problem exists
Dedicated 2 paragraphs for the problem description
© Mai Elshehaly

16. Next paragraph
© Mai Elshehaly

17. Why the authors chose and are able to write this paper
This is not necessary in many types of papers (e.g. design studies)
but because this is a research methodology paper, the readers need to be convinced that the authors are qualified and experienced enough to write such insights.
© Mai Elshehaly

18. Contribution
This paper aims to give the proper guidance on conducting design studies (to solve the problem).
The paper presents:
Definitions for design studies
A methodological framework for conducting design studies
Practical guidelines for design studies
© Mai Elshehaly

19. Approach (expands upon contributions)
© Mai Elshehaly

20. Contribution and Approach
In Abstract:
They started with contributions
They expanded contributions into a summary of approach
In Introduction:
They wrote a summary of approach
They concluded the Introduction section with contributions
© Mai Elshehaly

21. © Mai Elshehaly

22. Contribution and Approach
In Abstract:
They started with contributions
They expanded contributions into a summary of approach
In Introduction:
They wrote a summary of approach
They concluded the Introduction section with contributions
Clearly you can change this order according to your needs
© Mai Elshehaly

23. Related Work
© Mai Elshehaly

24. Related Work
You need to tell the readers that you are aware of other people’s efforts to tackle the same problem
You can cite one or two classic papers but the majority of your references in this section need to be up to date
You want to tell where these existing works fall short because this gives rationale for you to work on the problem and offer new contributions
© Mai Elshehaly

25. Now to the details
We already learned the outlines from the Abstract & Introduction
Time to delve into the details of the study
© Mai Elshehaly

26. Mapping Contributions to Sections
Definitions
A methodological framework
Practical guidelines
© Mai Elshehaly

27. What is a Design Study? A form of problem-driven research
The goal is to work with real users to solve real-world problems
A project in which vis. Researchers:
Analyze a specific real-world problem faced by domain experts
Design a visualization system that supports solving this problem
Validate the design
Reflect about lessons learned in order to refine visualization design guidelines
© Mai Elshehaly

28. What is NOT a design study?
Technique-driven research in which the goal is to “develop new and better techniques without necessarily establishing a strong connection to a particular documented user need.”
© Mai Elshehaly

29. Components for an accepted design study paper: (Look for these to review papers in our discussions)
Analysis
Real-world problem
Design
Validation
Reflection
Collaboration
© Mai Elshehaly

30. Components for an accepted design study paper: (Look for these to review papers in our discussions)
Analysis
Real-world problem
Design
Validation
Reflection
Collaboration
Translate tasks and data from domain-specific form into abstractions that a user can address through visualization
© Mai Elshehaly

31. Components for an accepted design study paper: (Look for these to review papers in our discussions)
Analysis
Real-world problem
Design
Validation
Reflection
Collaboration
Real users and real data are mandatory
© Mai Elshehaly

32. Components for an accepted design study paper: (Look for these to review papers in our discussions)
Analysis
Real-world problem
Design
Validation
Reflection
Collaboration
How to map tasks and data to visual representations
Need to show different possibilities
Select the best choice
© Mai Elshehaly

33. Components for an accepted design study paper: (Look for these to review papers in our discussions)
Analysis
Real-world problem
Design
Validation
Reflection
Collaboration
Justify your design according to known principles
Qualitative analysis of results
Informal expert feedback
Post deployment field studies
© Mai Elshehaly

34. Components for an accepted design study paper: (Look for these to review papers in our discussions)
Analysis
Real-world problem
Design
Validation
Reflection
Collaboration
You have a new software design but is it research?
Your design becomes research when you reflect on your design to improve the process of design itself
Examples:
Confirm guidelines
Refine guidelines
Reject guidelines
Propose new guidelines
© Mai Elshehaly

35. Components for an accepted design study paper: (Look for these to review papers in our discussions)
Analysis
Real-world problem
Design
Validation
Reflection
Collaboration
The role of domain experts is crucial
Attempts to simply apply techniques without a thorough understanding of the domain context can fail dramatically
© Mai Elshehaly

36. Types of contributions for an accepted design study paper: (Look for these to review papers in our discussions)
Problem characterization and abstraction
Validated visualization design
Reflection
© Mai Elshehaly

37. Reviewers’ Expectations
© Mai Elshehaly

38. Mapping Contributions to Sections
Definitions
A methodological framework
Practical guidelines
© Mai Elshehaly

39. The 9-stage Framework
© Mai Elshehaly

40. Learn Read a lot in visualization literature
This informs all later stages
© Mai Elshehaly

41. Winnow: select collaborators
Do they have real data? Can we have the data?
Will they be available for meetings?
Is there an interesting vis research question in this problem? (from the learn stage)
Is there a real need or are existing approaches good enough?
© Mai Elshehaly

42. Cast: Identify collaborator roles
Front-line analyst
Project gatekeeper (can stop the project or stop us from getting data)
Fellow tool builder
© Mai Elshehaly

43. Discover: (Requirements analysis) Problem Characterization and Abstraction
Learn about the target domain: practices, needs, problems, and requirements
Identify if and how visualization can lead to insight
Identify the successful parts of the workflow (that do not need vis) and the problematic parts (that can benefit from our vis research)
© Mai Elshehaly

44. Discover: (Requirements analysis) Problem Characterization and Abstraction
The domain expert speaks and the vis researcher listens
The vis researcher abstracts then elicits feedback from the expert on the abstraction
The vis researcher continues to refine the abstraction based on expert’s feedback
© Mai Elshehaly

45. Problem Characterization
A straightforward description of the problem that is easily understandable by computer scientists and people in the VIS community
A mathematical representation of the data model
A semi-formal description of the tasks and their interaction components
© Mai Elshehaly

46. Challenges
Most users do not accurately describe their analysis and visualization needs
Many only use visualization to present final results but they are not aware of the benefits of visual analytics
© Mai Elshehaly

47. Methods for the discover phase
Interviews
Observations: fly-on-the-wall technique
Contextual inquiries
Reading domain literature
© Mai Elshehaly

48. Design: Data Abstraction, Visual Encoding & Interaction
Comes after reaching a shared understanding of a problem with domain experts
Main activities in this phase:
Generation of data abstraction, visual encodings, and interaction mechanisms
Validation of data abstractions, visual encodings, and interaction mechanisms
Further refinements of the problem characterization and data abstraction
Create a broad consideration space of possible solutions (set of data abstractions, visual encodings, and interaction mechanisms)
Filter the broad consideration space to a narrow proposal space base don design principles and guidelines (
Present the solutions in the proposal space to domain experts in the form of paper mockups, data sketches, or low-level prototypes
© Mai Elshehaly

49. Implement: Prototypes, Tool & Usability
Integrate your software with the domain experts’ existing workflow
Follow software engineering and HCI guidelines:
Rapid prototyping: get user feedback, throw away code that doesn’t fit the need
Usability engineering: if your tool is too difficult to use, it will fail
© Mai Elshehaly

50. Deploy: Release & Gather Feedback
Find out if domain experts are truly helped by the new solution:
Do they perform tasks faster?
Do they perform tasks more correctly?
Do they have a less workload?
Are they able to do things they were not able to do before?
© Mai Elshehaly

51. Reflect: Confirm, Refine, Reject, Propose Guidelines
How dis your study relate to the larger research area of visualization?
How does it allow other researchers in the field to benefit from your findings?
Does it improve or refine existing design guidelines?
Does it confirm these guidelines or reject them?
Does it propose new guidelines?
© Mai Elshehaly

52. Write: Design Study Paper
You can start writing at any point
We will start writing as soon as we identify the problem
Avoid writing too much detail on the domain knowledge you acquired
Put enough effort into writing extremely clear abstractions of the tasks and data (only include enough domain knowledge to understand these abstractions)
© Mai Elshehaly

53. Conclusion: Take home message
© Mai Elshehaly