Augmenting Intellect Professor Michael Terry February 6, 2007

CS497 / 2 Talk Overview Who I am Research interests Time lapse detour Overview of Human Computer Interaction (HCI) Augmenting human intellect –History –Challenges –Ill-defined problems –Interface-level support –Open problems

CS497 / 3 Talk Protocol You should be actively asking questions…

CS497 / 4 Who I Am One of your professors specializing in Human Computer Interaction (HCI) –Ed Lank and Bill Cowan also specialize in this area More about HCI at Waterloo – Have taught CS 349 (User Interfaces) and CS 489 (Human Computer Interaction) –CS 349 teaches mechanics of constructing interactive applications –CS 489 teaches process of discovering what to build and whether design is useful, usable

CS497 / 5 Research Interests Computational support for creative practices –Contextualized and realized within open source software Novel time lapse visualization techniques Both include elements of information visualization

CS497 / 6 Time for that Detour…

CS497 / 7 Time Lapse: Scales of Time Normally perceive world sequentially –Second-by-second, minute-by-minute Continuous barrage of stimuli to filter Easy to miss long-term trends and events –Example: Patterns of clapping Visualizations can help us discover these trends and events

CS497 / 8 Time Maps

CS497 / 9 Time Maps

CS497 / 10 Time Maps

CS497 / 11 Current Work: News Images Goals –Collapse days, months of news images into single, comprehensible visualization –Enable viewer to “feel” flow of news over time –Identify patterns, trends, biases not easily discovered otherwise Challenges –No constants between images, like traditional time lapse Undergrad URAs: –Ilya Grigorik –Tom Ayre –Ray Kuo

Credit: Ilya Grigorik

Yahoo News BBC News

Credit: Tom Ayre

CS497 / 17 Switching Gears… Overview of HCI –Types of problems investigated –Goals of HCI Augmenting intellect –Historical roots –Challenges –Ill-defined problems –Interface-level support –Open problems

CS497 / 18 What is HCI? HCI!

CS497 / 19 HCI’s Concerns What to build? For whom? In what form? What should be the language of interaction? How well does it meet its goals? What are the local and global impact of the choices made?

CS497 / 20 Example: Remote Communication

CS497 / 21 HCI is Interdisciplinary Computer scientists Psychologists Anthropologists Industrial designers Graphic designers Electrical engineers Artists Has implications in academia and industry…

CS497 / 22 Goals of HCI Make computational artifacts that are “useful, usable, used” (Dix, Finlay, Abowd, Beale 2004) Useful: Tool accomplishes what is required Usable: Performs task “easily and naturally” Used: People want to use it

CS497 / 23 More Goals of HCI Safe and forgiving –Mission critical systems, such as air traffic control, airplane cockpits, power plants, weapons systems… Universally accessible –Computers can compensate for physical and cognitive impairments Quick to learn to use –And to remember how it should be used… Efficient

CS497 / 24 Avoid Imprecision! People often cite “easy to use” and “usable” as laudable goals of interface design But how can we measure “easy to use” or “usable”? Terms mean many things to many different people –Not directly measurable Saying a system is “easy to use” or “usable” does not tell us why users perceive it to be that way –This makes it difficult to replicate successes and avoid similar mistakes These points will be important later…

CS497 / 25 Be Specific and Prioritize Efficiency, learnability, desirability, cognitive load, physical exertion are all worthwhile measures for considering a system’s design… But first concern should always be usefulness of system

CS497 / 26 Why HCI? Why should we be concerned with HCI? Isn’t it enough to practice the Darwinian evolution of user interfaces? –Survival of the fittest? –Recombination of the best features into new interactive systems? (Insert your own Mac OS X / Vista joke here…) Computation has the potential to be one of the most significant complements to our human intellect…

CS497 / 27 “There is a growing mountain of research. But there is increased evidence that we are being bogged down today as specialization extends. The investigator is staggered by the findings and conclusions of thousands of other workers — conclusions which he cannot find time to grasp, much less to remember, as they appear.” When was this statement made?

CS497 / 28 Vannevar Bush Headed “Office of Scientific Research and Development” in US –Manhattan project, other WWII science efforts –Oversaw 6000 scientists 1945 article in The Atlantic inspires computer scientists to present day Goal was to augment human intellect

CS497 / 29 Vannevar Bush “A memex is a device in which an individual stores all his books, records, and communications… It is an enlarged intimate supplement to his memory.” Proposes associative links between content Dual display setup! Direct annotation of stored content Proposes direct connection to nervous system

CS497 / 30 Douglas Engelbart “By ‘augmenting human intellect’ we mean increasing the capability of a man to approach a complex problem situation, to gain comprehension to suit his particular needs, and to derive solutions to problems… [We seek] more-rapid comprehension, better comprehension, the possibility of gaining a useful degree of comprehension in a situation that previously was too complex, speedier solutions, better solutions, and the possibility of finding solutions to problems that before seemed insoluble…” Augmenting Human Intellect, 1962 SRI Report

CS497 / 31 Douglas Engelbart Creates the mouse, chording keyboard Demonstrates concepts such as copy/paste, computer-supported collaborative work in a 1968 live demo “Eats own dog food” as method of assessment

CS497 / 32 Beyond the Knowledge Worker Ivan Sutherland’s Sketchpad (1963) –Light pen –Direct manipulation –Early “WYSIWYG” Expands computer domain to include artists, draftsmen, and more Language of interface moves substantially closer to task domains –How and why?

CS497 / 33 Sketchpad Video

CS497 / 34 “…[Sketchpad] was a model for totally new ways of operating computers; by changing something on the display screen, it was possible, via Sketchpad, to change something in the computer’s memory.” - Howard Rheingold in Tools for Thought

CS497 / 35 Significance of Early Work Early work recast digital data in higher-level representations that facilitate its: –Comprehension –Creation –Manipulation –Access Consider Sketchpad… Higher-level representations make use of higher-level cognitive capabilities (e.g., innate perceptual skills) Fast forward to today…

CS497 / 36 Today… Computers can fit on: –Desktop

CS497 / 37 Today… Computers can fit on: –Desktop –In our hands

CS497 / 38 Today… Computers can fit on: –Desktop –In our hands –On our wrists

CS497 / 39 Thad Starner Today… On our glasses…

CS497 / 40 Today… In our clothes Pioneer’s Embedded Display MIT’s Stitched Number Pad

CS497 / 41 Today… Hard disks can store lifetimes of memories –See “MyLifeBits” by Gordon Bell at MS –1 terabyte to hold our 20 th century lives? Search engines put volumes of information at our fingertips –When is the last time you used a card catalog? Computation is cheap –$100’s for basic computational machinery, ~$1/GB for storage CPUs can process billions and billions of instructions every second Computation can be packaged in forms both large and small But…

CS497 / 42 It’s not enough to have faster, more capable machines…

CS497 / 43 Source: wii_PS3_061115/ ?hub=Specials&pr=0

CS497 / 44 How can we fruitfully and reliably augment human intellect with all of this computational power?

CS497 / 45 Your Answers…? How can computation augment human intellect?