Ben Shneiderman Founding Director ( ), Human-Computer Interaction Laboratory Professor, Department of Computer Science Member, Institute for Advanced Computer Studies & Institute for Systems Research University of Maryland College Park, MD UIST Vancouver November 3, 2003 Creativity Support Tools: A Grand Challenge for Interface Designers Ben Shneiderman Founding Director ( ), Human-Computer Interaction Laboratory Professor, Department of Computer Science Member, Institute for Advanced Computer Studies & Institute for Systems Research University of Maryland College Park, MD UIST Vancouver November 3, 2003

Human-Computer Interaction Laboratory Interdisciplinary research community - Computer Science & Psychology - Information Studies & Education

User Interface Design Goals   Cognitively comprehensible: Consistent, predictable & controllable   Affectively acceptable: Mastery, satisfaction & responsibility Design philosophy: Direct Manipulation NOT: Adaptive, autonomous & anthropomorphic

Scientific Approach (beyond user friendly)  Specify users and tasks  Predict and measure  time to learn  speed of performance  rate of human errors  human retention over time  Assess subjective satisfaction (Questionnaire for User Interface Satisfaction 7.0)  Accommodate individual differences  Consider social, organizational & cultural context

Scientific Approach (beyond user friendly)  Specify users and tasks  Predict and measure  time to learn  speed of performance  rate of human errors  human retention over time  Assess subjective satisfaction (Questionnaire for User Interface Satisfaction 7.0)  Accommodate individual differences  Consider social, organizational & cultural context

Library of Congress  Scholars, Journalists, Citizens  Teachers, Students

Visible Human Explorer (NLM)  Doctors  Surgeons  Researchers  Students

NASA Environmental Data  Scientists  Farmers  Land planners  Students

Bureau of Census  Economists, Policy makers, Journalists  Teachers, Students

NSF Digital Government Initiative  Find what you need  Understand what you Find  Understand what you Find UMd & UNC

Creativity Support Tools Creativity Support Tools

Creativity Support Tools: Goals   More people, more creative, more of the time   Revolutionary breakthroughs, paradigm shifts, H-creativity   Evolutionary, normal science, music & art, creative knowledge work   Impromptu everyday creativity   Raised expectations for professionals   Tailored solutions   Cheaper, faster, better

Structuralists: A plan, method, process   Polya's four steps in How to Solve It (1957):   1) Understanding the problem   2) Devising a plan   3) Carrying out the plan   4) Looking back   Couger (1996) reviews 22 "creative problem solving methodologies"   Preparation   Incubation   Illumination   Verification

Structuralists: A plan, method, process   Atman's design steps:   Problem definition – identify need   Gather information   Generate ideas – brainstorm & list alternatives   Modeling – describe how to build   Feasibility Analysis   Evaluation – compare alternatives   Decision – select one solution   Communication – write or present to others   Implementation (Atman et al., Design Thinking Research Symposium2003)

Inspirationalists: Aha, Aha, Aha!   Free associations   Brainstorming   Thesauri, photo collages   Random stimuli, inkblots   Breaking set   Getting away to different locations   Working on other problems   Meditating, sleeping, walking   Visualization   2-d networks of ideas   Sketching

Situationalists: context, community, collaboration   Personal history   Family history, parents, siblings   Challenging teachers, inspirational mentors   Supportive peers and partners   Consultation   Peers and mentors   Early, middle and late stages   Information and empathic support   Motivations   Fame, legacy, admiration   Competition

Csikszentmihalyi’s book Creativity (1993)   1) Domain : e.g. mathematics or biology "consists of a set of symbols, rules and procedures”   2) Field : "the individuals who act as gatekeepers to the domain...decide whether a new idea, performance, or product should be included”   3) Individual : creativity is "when a person... has a new idea or sees a new pattern, and when this novelty is selected by the appropriate field for inclusion in the relevant domain"

Genex (Generator of Excellence) Framework Genex (Generator of Excellence) Framework   Collect   Learn from previous works in digital libraries, the web, etc.   Relate   Consult with peers & mentors, early, mid & late stages   Create   Explore, discover, compose, evaluate possible solutions   Donate   Disseminate refined results and contribute to the digital libraries, the web, etc. (Codex, memex, genex: The pursuit of transformational technologies IJHCI, 1998)

Genex: Some potential down sides Genex: Some potential down sides   Collect   Will knowledge of previous work limit imagination?   Relate   Could mentors discourage exotic ideas?   Could peers rip-off your innovation?   Create   Will using standard tools limit creativity?   Donate   Could the desire for intellectual property protection limit dissemination?

Eight Activities in Genex   1) Searching & browsing digital libraries   2) Consulting with peers & mentors   3) Visualizing data & processes   4) Thinking by free associations   5) Exploring solutions - What if tools   6) Composing artifacts & performances   7) Reviewing & replaying session histories   8) Disseminating results (Creating creativity: User interfaces for supporting innovation ACM TOCHI, 3/2000)

1) Searching & Browsing Digital Libraries   Effective search: Basic Google Search   Improved multimedia search   Overviews & previews   Result set categorization & visualization   Multiple session searches (Clarifying Search, CACM 4/98)

1) Advanced Search: Google

1) Search: Overviews & Previews   Faceted search   Preview cardinality of result sets

2) Consulting with Peers & Mentors   Early, middle and late stages   Information and empathic support   Build trust by negotiated expectations   , listservs, newsgroups, discussion boards   Chat rooms, instant messaging, audio/video conferencing   Comprehensive online communities   Tele-medicine, tele-meeting, tele-democracy   Collaboratories

2) Components of Negotiated Expectations   Clearly identify and refine through dialog   who I am   what I want to do   Declare understandings   Why I think you can help   How you can help (specific request with time period)   How much is in it for you (payment, shared honor, appreciation)

2) Example Request for Consultation   Poor: Dear Prof. Shneiderman: Attached is my PhD proposal, please tell me what you think.   Better: Dear Prof. Shneiderman: I am a Computer Science PhD student at Imperial College. My advisor, Prof. Spence, knows you and I have read your papers. My work extends your concept of dynamic queries. I’ve attached my 2-page doctoral proposal in the hopes that you can give me your comments before I defend it in two weeks. Your input would be acknowledged and Dr. Spence would be glad to pay for your plane ticket to join the committee meeting.

3) Visualizing Data & Processes   Visual bandwidth is enormous   Human perceptual skills are remarkable   Trend, cluster, gap, outlier...   Color, size, shape, proximity...   Human image storage is fast and vast   Opportunities   Spatial layouts & window coordination   Information visualization   Scientific visualization and simulation   Telepresence and augmented reality   Virtual Environments

3) Visualizing Data & Processes   Visual bandwidth is enormous   Human perceptual skills are remarkable   Trend, cluster, gap, outlier...   Color, size, shape, proximity...   Human image storage is fast and vast   Opportunities   Spatial layouts & window coordination   Information visualization   Scientific visualization and simulation   Telepresence and augmented reality   Virtual Environments

Treemap Implementations

3) TimeSearcher   Time series   Stocks   Diseases   Weather   Genes   User-specified patterns   Rapid search

3) Information Visualization: Mantra   Overview, zoom & filter, details-on-demand

3) Information Visualization: Data Types   1-D Linear Document Lens, SeeSoft, Info Mural, Value Bars   2-D Map GIS, ArcView, PageMaker, Medical imagery   3-D World CAD, Medical, Molecules, Architecture   Multi-Dim Parallel Coordinates, Spotfire, XGobi, Visage, Influence Explorer, TableLens, DEVise   Temporal Perspective Wall, LifeLines, Lifestreams, Project Managers, DataSpiral   Tree Cone/Cam/Hyperbolic, TreeBrowser, Treemap   Network Netmap, netViz, SeeNet, Butterfly, Multi-trees (Online Library of Information Visualization Environments) otal.umd.edu/Olive

4) Thinking by Free Associations  Free associations  Brainstorming  Thesauri, photo collages  Random stimuli, inkblots  Breaking set  Getting away to different locations  Working on other problems  Meditating, sleeping, walking  Visualization  2-d networks of ideas  Sketching

An Idea Visualization Tool The human vision is by far the most developed and powerful faculty. The Idea Processor exploits visual attributes such as: color, shape, size, scale, position, depth, link, icon, etc. Visual cues facilitate recall, association, and discovery. Diagrams and pictures help you to represent and solve complex problems. Visualization reinforces your short term memory. Towards Higher Abstractions Ideas and diagrams are the basic abstractions of the Axon Idea Processor. Ideas are shown as graphical objects and its relationship shown as links. You get the big picture at all times, and details can be hidden from view. Stimulate Recall & Creativity The Idea Processor has an integrated Checklist Management System and a library of Checklists on problem-solving strategies, words of wisdom, etc. Checklists are effective means of capturing and transferring knowledge, and it effectively amplifies your intelligence.

5) Exploring Solutions - What If Tools   State space exploration   “Combinationist theory”   Spreadsheets   Simulation as a third paradigm of science   SimCity & Flight Simulator   Economic models   Weather forecasts

5) Exploring Solutions - What If Tools   Terry & Mynatt: Previews

6) Composing Artifacts & Performances  Initiate a new composition  Exemplars  Templates  Processes  Revise at multiple levels  Low  Middle  High  Evaluate and refine  Feedback about problems  Measurement (Composition, Hawaii Int’l Conf. on Systems Science, January 2000)

6) Composing Artifacts & Performances  Initiate a new composition

7) Reviewing & Replaying Session Histories  Record compact histories  Allow users to review & annotate history  Disseminate histories (histories as first class objects)  Send by  Post to website  Consult synchronously & asynchronously  Edit, extract, combine, search  Replay: slow, fast, reverse  Macros to automate exploration (Photoshop) (Learning Histories, Computer Supported Collaborative Learning Conf., December 1999)

7) Imaginary dialysis procedure exploration

7) Adobe Photoshop History tools

8) Disseminating Results   Disseminate refined solution to gatekeepers   Facilitate web publishing & focused advertising   Reach subscribers & organizational gatekeepers   Ensure quality by editors & reviewers   Contribute to digital library communities   Journals, books, resources, libraries, communities   Science fairs, student projects, galleries, performances

Genex: Integrated Framework for Software   Modular design to allow components   Common file formats to ease data movement   Consistent commands to reduce cognitive load   Shortcuts for experts & immediate feedback   Direct manipulation for rapid learning, powerful actions, low error rates, high retention

Genex Phases and Activities

Challenges for Creativity Researchers   Domain knowledge is vital   Creative work may take years   Individuals have highly varied approaches   Evaluation is difficult   Controlled studies are unrealistic   Case studies are not replicable   Theories are shallow

An Inspirational Muse: Leonardo da Vinci ( ) Renaissance Man   Combined science & art   Integrated engineering & esthetics   Balanced technology advances & human values   Merged visionary & practical (MIT Press, Oct 2002)

Human-Computer Interaction Laboratory 21 th Annual Symposium June 3-4,

References Shneiderman, Ben and Plaisant, Catherine, Designing the User Interface: Strategies for Effective Human-Computer Interaction: Fourth Edition, Addison Wesley Longman, Boston, MA (2004) Shneiderman, Ben, Relate-Create-Donate: A teaching philosophy for the cyber-generation, Computers & Education 31, 1 (1998), Shneiderman, B., Creating Creativity: User interfaces for supporting innovation, ACM Transactions on Computer Human Interaction 7, 1 (2000), Shneiderman, B., Supporting creativity with powerful composition tools for artifacts and performances, Proc. 33 rd Hawaii International Conference On System Sciences (HICSS) (1/2000).