History of GUI John Kelleher

Why study HCI’s history? Understanding where you’ve come from can help a lot in figuring out where you’re going - repeat positive lessons “Those who don’t know history are doomed to repeat it” - avoid negative lessons Knowledge of an area implies an appreciation of its history

Context – Computing in 1945 Harvard Mark I 55 feet long, 8 feet high, 5 tons Ballistics calculations Physical switches (bef. microprocessor) Paper tape Simple arithmetic and fixed calculations (before programs) 3 seconds to multiply

Context – Computing in 1945 First computer bug (Harvard Mark II) Grace Murray Hopper Physical nanoseconds

Vannevar Bush (1890-1974) “As we may think” - Atlantic Monthly, 1945 Faculty member MIT Coordinated WWII effort with 6000 US scientists Social contract for science Federal government funds universities Universities do basic research Research helps economy and national defense Identified the information storage and retrieval problem: new knowledge does not reach the people who could benefit from it “The World has arrived at an age of cheap complex devices of great reliability; and something is bound to come of it.” “publication has been extended far beyond our present ability to make real use of the record”

Bush’s Memex Conceiving Hypertext and the World Wide Web a device where individuals stores all personal books, records, communications etc Even contemplated wearable cameras (see work at MIT Media Lab) Encyclopedia Britannica for a nickel Automatic transcripts of speech items retrieved rapidly through indexing, keywords, cross references,... can annotate text with margin notes, comments... can construct a ‘Trails of discovery’ (a chain of links) through the material and save it acts as an external memory! Direct capture of nerve impulses! Bush’s Memex device based on microfilm records, not computers! but not implemented Picture from http://www.dynamicdiagrams.com/design/memex/model.htm#download Paper: http://www.theatlantic.com/unbound/flashbks/computer/bushf.htm

Context – Computing in the 1960s Transistor (1948) Modern P4 has about 15 million transistors (size of fingernail) ARPA (1958) Advanced Research Projects Agency Founded immediately post-Sputnik Budget of only several million$ Modern DARPA budget about $2 Billion (2001) Modern NSF budget about $4.5 Billion (2001) Timesharing (1950s) Terminals and keyboards Computers still primarily for scientists and engineers

J.C.R. Licklider (1915-1990) “Lick” became director of ARPA in 1962. With ARPA sponsorship, the first CS programs were created: MIT, CMU, Berkeley, Stanford Did self-observation of his daily work. Observed that much work was mundane and related to accessing and organizing information Proposed: Digital libraries Display screens with pen input and character recognition Wall displays for collaborative work Speech recognition and production for HCI Outlined “man-computer symbiosis” “The hope is that, in not too many years, human brains and computing machines will be coupled together very tightly and that the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information-handling machines we know today.”

Licklider’s Goals Produced goals that are pre-requisite to “man-computer symbiosis” immediate goals: time sharing of computers among many users electronic i/o for the display and communication of symbolic and pictorial information interactive real time system for information processing and programming large scale information storage and retrieval intermediate goals: facilitation of human cooperation in the design & programming of large systems combined speech recognition, hand-printed character recognition & lightpen editing long term visions: natural language understanding (syntax, semantics, pragmatics) speech recognition of arbitrary computer users heuristic programming

Ivan Sutherland (1938-) 1963 PhD MIT - SketchPad Helped head DARPA Info Processing Now a VP and Sun Fellow SketchPad - Sophisticated drawing package introduced many new ideas/concepts now found in today’s interfaces hierarchical structures defined pictures and sub-pictures object-oriented programming: master picture with instances constraints: specify details which the system maintains through changes icons: small pictures that represented more complex items copying: both pictures and constraints input techniques: efficient use of light pen world coordinates: separation of screen from drawing coordinates recursive operations: applied to children of hierarchical objects Parallel developments in hardware: “low-cost” graphics terminals input devices such as data tablets (1964) display processors capable of real-time manipulation of images (1968)

Douglas Engelbart (1925-) Strongly influenced by Bush’s article Stanford Research Institute (SRI) Augmentation Research Center 1962 Paper "Conceptual Model for Augmenting Human Intellect" “...increasing the capability of a man to approach a complex problem situation, gain comprehension to suit his particular needs, and to derive solutions to problems....”

Douglas Englebart 1968: NLS (oNLine System) System, Fall Joint Computer Conference (SF) document processing modern word processing hierarchical hypertext multimedia (mixed text and graphics) input/output the mouse one-handed corded keyboard high resolution display view control (and windows) specially designed furniture shared work shared files personal annotations electronic messaging desktop conferencing (video/audio managed by computer) shared displays (what you see is what I see) multiple pointers user testing to see how people worked, need for training... All took place before Unix and C (1970s), ARPAnet (1969) and later Internet

Douglas Englebart & NLS "If ease of use was the only valid criterion, people would stick to tricycles and never try bicycles." - Engelbart "I tell people: look, you can spend all you want on building smart agents and smart tools…" "I'd bet that if you then give those to twenty people with no special training, and if you let me take twenty people and really condition and train them especially to learn how to harness the tools…" "The people with the training will always outdo the people for whom the computers were supposed to do the work." “At SRI in the 1960s we did some experimenting with a foot mouse. I found that it was workable, but my control wasn't very fine and my leg tended to cramp from the unusual posture and task.”

Alan Kay (1940-) Ph.D. 1969 (Utah) Computer Graphics In 1968, met Seymour Papert (LOGO) in the MIT AI Lab. - kids can program! Moved to Xerox PARC in 1972 Started developing “Smalltalk”, in the Learning Research Group First general OO programming language Influenced by Simula Atari – Chief Science Officer Now a Disney Fellow "By the time I got to school, I had already read a couple hundred books. I knew in the first grade that they were lying to me because I had already been exposed to other points of view. School is basically about one point of view -- the one the teacher has or the textbooks have. They don't like the idea of having different points of view, so it was a battle. Of course I would pipe up with my five-year-old voice."

Alan Kay Dynabook vision (and cardboard prototype) of a notebook computer (conceived in 1968) “Imagine having your own self-contained knowledge manipulator in a portable package the size and shape of an ordinary notebook. Suppose it had enough power to out-race your senses of sight and hearing, enough capacity to store for later retrieval thousands of page-equivalents of reference materials, poems, letters, recipes, records, drawings, animations, musical scores...” Kay develops the Xerox Alto (1972)1 and Star (1981), the first real PCs "The best way to predict the future is to invent it" 1 Primary hardware developers: Butler Lampson & Chuck Thacker

Seymour Papert Get children to program as a technique for learning Learn by doing Logo Professor MIT Media Lab Lego Mindstorms

Xerox PARC Created in 1970 Some early inventions "Architecture of information“ Camelot Era Some early inventions Ethernet Networking Laser Printer Desktop Computing These ideas seem ordinary today Measure of success

Xerox Alto First Personal Computer 1972 Precursor to Xerox Star local processor, bit-mapped display, mouse modern graphical interfaces text and drawing editing, electronic mail windows, menus, scroll bars, mouse selection, etc local area networks (Ethernet) for personal workstations could make use of shared resources 1972 Precursor to Xerox Star Internal only to PARC $45,000 / PC ALTAIR 8800 (1975) Popular electronics article that showed people how to build a computer for under $400 Seed of Microsoft sown

Xerox Star Commercial PC 1981 $16,500 / PC First commercial personal computer designed for “business professionals” First comprehensive GUI used many ideas developed at Xerox PARC familiar user’s conceptual model (simulated desktop) promoted recognizing/pointing rather than remembering/typing property sheets to specify appearance/behaviour of objects what you see is what you get (WYSIWYG) small set of generic commands that could be used throughout the system high degree of consistency and simplicity modeless interaction limited amount of user tailorability

Screen shot of Xerox Star

Xerox Star – Property Sheets

Xerox Star First system based upon usability engineering inspired design extensive paper prototyping and usage analysis usability testing with potential users iterative refinement of interface Commercial failure cost ($15,000) IBM had just announced a less expensive machine limited functionality - e.g., no spreadsheet closed architecture 3rd party vendors could not add applications perceived as slow but really fast! slavish adherence to direct manipulation Steve Jobs and Apple engineers visited PARC in 1979, and that set the path for Apple 15 PARC engineers migrated to Apple Pictures from http://www.geocities.com/SiliconValley/Office/7101 Desktop metaphor Windows / Icons / Menus / Pointer WYSIWYG

Early Personal Computers 1997 Apple II 1979 VisiCalc - “killer app” for Apple II 1981 IBM XT/AT http://apple2history.org/history http://www.bricklin.com/history/saiearly.htm http://www.vintagecomputers.freeserve.co.uk/ibmpc/

Apple Macintosh - 1984 Aggressive pricing Good interface guidelines $2500 Good interface guidelines Third party applications Great graphics, laser printer

Apple (1981) Apple Lisa (1983) Apple Macintosh (1984) based upon many ideas in the Star; predecessor of Macintosh, somewhat cheaper ($10,000) commercial failure as well Apple Macintosh (1984) “old ideas” but well done! ‘Mac’ succeeded because: aggressive pricing ($2500) did not need to trailblaze learnt from mistakes of Lisa and corrected them; ideas now “mature” market now ready for them developer’s toolkit encouraged 3rd party non-Apple software interface guidelines encouraged consistency between applications domination in desktop publishing because of affordable laser printer and excellent graphics