1. Human-Computer Interaction Lecture 2: HCI History
Key people, events, ideas and
paradigm shifts
Shandong University
This material has been developed by Georgia Tech HCI faculty, and continues to evolve. Contributors include Gregory Abowd, Jim Foley, Diane Gromala, Elizabeth Mynatt, Jeff Pierce, Colin Potts, Chris Shaw, John Stasko, and Bruce Walker. This specific presentation also borrows from James Landay and Jason Hong at UC Berkeley.

2. Series of technological advances Series of paradigm shifts
The Evolution of HCI
Series of technological advances
lead to and are sometimes facilitated by a
Series of paradigm shifts
that in turn are created by a
Series of key people and events

3. Why study HCI’s history?
Understanding where you’ve come from can help a lot in figuring out where you’re going - repeat positive lessons
“Those who don’t know history are doomed to repeat it” - avoid negative lessons
Knowledge of an area implies an appreciation of its history

4. What are Paradigms
Predominant theoretical frameworks or scientific world views
e.g., Aristotelian, Newtonian, Einsteinian (relativistic) paradigms in physics
Understanding HCI history is largely about understanding a series of paradigm shifts
Not all listed here are necessarily “paradigm” shifts, but are at least candidates
History will judge which are true shifts

5. Howard Rheingold – Tools for Thought
History of interactive breakthroughs
On-line at
One of several good sources

6. (Some of the) Key Technological Advances
Starting point
Computing in 1945
Batch processing
Interactive graphics systems
Time sharing computers
One computer to many people
Internet

7. More Key Technological Advances
The desktop / personal computer
One computer to one person
Inexpensive, low-power chips
Many computers to one person
Wireless connectivity

8. Paradigm Shifts – How We Use Computers
Interactive Computing - time sharing, Basic
WIMP Interfaces
Windows, Icons, Mouse, Pointing
Direct Manipulation
Metaphors
Hypertext / WWW
Computers for person-to-person communications – not just for computing
, CSCW

9. Multimodal interfaces
More Paradigm Shifts
Multimodal interfaces
Mode is a human communication channel
Not just the senses e.g., speech and non-speech audio are two modes
Emphasis on simultaneous use of multiple channels for I/O
Immersive (VR) interfaces
Mobile computing
Ubiquitous computing

10. (Some of the) Key People and Events
Vannevar Bush
Doug Engelbart
Ivan Sutherland
J. R. (Lick) Licklider
Alan Kay
Ted Nelson
Nicholas Negroponte
Mark Weiser
Jaron Lanier
Events
Founding of Xerox PARC
Lisa / Macintosh

11. Key Technological Advances Key Paradigm Shifts Key People and Events
Telling the Story
Key Technological Advances
Key Paradigm Shifts
Key People and Events
Interleaved in more or less chronological order

12. Digital computer grounded in ideas from 1700’s & 1800’s
In the Very Beginning
Digital computer grounded in ideas from 1700’s & 1800’s
Technology became available in the 1940’s and 1950’s
Movie: Inception/ Minority Report

13. History of HCI
Mechanical Computers
1623 Schickard makes "Calculating Clock". 6-digit machine can add, subtract, bell indicates overflow.
1674 Leibniz designs his "Stepped Reckoner” Can multiply, with operands of up to 5 and 12 digits. User turns a crank for each unit in each digit
1820 de Colmar makes "Arithmometer” First mass-produced calculator. Does multiplication & division. It is also the most reliable calculator yet. Continue to be sold for about 90 years.
1889 Felt invents the first printing desk calculator.
1935 IBM introduces "IBM 601", punch card machine capable of 1 multiplication /second are made.
1945 Mauchly & Eckert "ENIAC” for ballistics. 30 tons, 1000 ft2 of floor, 140 kilowatts of electricity, 17,468 vacuum tubes
1600..
1623
Wilhelm Schickard makes his "Calculating Clock".
1644
Blaise Pascal makes his "Pascaline".
1668
Sir Samuel Morland, of England, produces a non-decimal adding machine.
1674
Gottfried Wilhelm von Leibniz designs his "Stepped Reckoner".
1700..
1775
Charles Earl Stanhope makes a multiplying calculator similar to Leibniz's.
1770-6
Mathieus Hahn also makes a successful multiplying calculator.
1786
J. H. Mueller conceives the idea of what came to be called a "difference engine".
1800..
1820
Charles Xavier Thomas de Colmar makes his "Arithmometer".
1822
Charles Babbage starts building his difference engine.
1832
Babbage and Clement produce a prototype segment of the difference engine.
1834
George Scheutz, of Stockholm, produces a small difference engine in wood.
Babbage conceives, and begins to design, his "Analytical Engine".
1842
Babbage's difference engine project is officially canceled.
1843
Scheutz and his son produce a 3rd-order difference engine with printer.
1847-9
Babbage designs an improved, simpler difference engine.
1853
The Scheutzes complete the first full-scale difference engine.
1858
The first two Tabulating Machines are sold.
1871
Babbage produces a prototype section of the Analytical Engine's mill and printer.
Charles Babbage dies.
1878
Ramon Verea invents a calculator with an internal multiplication table.
1879
A committee concludes that the Analytical Engine is impossible.
1885
A multiplying calculator enters mass production.
1886
Dorr E. Felt ( ), of Chicago, makes his "Comptometer".
1889
Felt invents the first printing desk calculator.
1890
US Census results are tabulated using Hollerith punch card tabulators.
1892
William S. Burroughs ( ) starts the office calculator industry.
1906
Henry Babbage completes the mill of his father's Analytical Engine.
1919
First flip-flop circuit design.
1930s
1931-2
E. Wynn-Williams constructs a binary digital counter.
1935
International Business Machines introduces the "IBM 601" punch card machine.
1937
George Stibitz constructs a demonstration 1-bit binary adder using relays.
Alan M. Turing publishes a paper on "computable numbers".
1938
Shannon publishes a paper on the implementation of symbolic logic using relays.
Zuse completes a prototype mechanical binary programmable calculator, the Z1.
1939
Zuse and Schreyer begin work on the "V2" (later " Z2").
John V. Atanasoff and Clifford Berry complete a prototype 16-bit adder.
1940s
1940
Williams and Stibitz complete a calculator with complex numbers.
Zuse is released from the army and completes the Z2.
1941
Atanasoff and Berry complete a calculator for simultaneous linear equations.
Zuse completes the "V3" (later "Z3").
1943
Howard H. Aiken and his team complete the "ASCC Mark I".
Max Newman, Wynn-Williams, and their team complete the "Heath Robinson".
Williams and Stibitz complete the "Relay Interpolator".
Tommy Flowers and his team at Bletchley Park complete the first "Colossus".
1944-5
Zuse almost completes his first full-scale machine, the "V4".
1945
Zuse invents a programming language called Plankalkuel.
John von Neumann describes what is later known as a von Neumann computer. The ENIAC is completed.
1946
The ENIAC is revealed to the public.
1947
Aiken and his team complete the " Harvard Mark II".
First actual case of bug being found.
Magnetic core memory is patented.
The magnetic drum memory is independently invented by several people.
1948
Wallace Eckert of IBM, with his team, completes the "SSEC".
Newman, Freddie C. Williams, and their team complete their "Mark I".
The ENIAC is improved, using ideas from Clipper and Metropolis.
IBM introduces the "IBM 604".
1949
Jay W. Forrester and his team at MIT construct the "Whirlwind".
Forrester conceives of magnetic core memory as it is to become common.
The Manchester Mark I acquires a secondary memory.
Maurice Wilkes (1913-) and his team complete the "EDSAC".
Presper Eckert and Mauchly complete the "BINAC" for the US Air Force.
Aiken's team completes the "Harvard Mark III".
1950s
1950
A group at the National Physical Laboratory, England, complete the "Pilot ACE".
Zuse's Z4 is finally completed and goes into service at the ETH Zurich.
Hartree estimates 3 computers will suffice to handle all calculations in England.
1951
Ferranti Ltd. completes the first commercial computer, yet another "Mark I".
Eckert and Mauchly complete the first "UNIVAC".
Thompson, Simmons, and their team complete the "LEO I".
Grace Murray Hopper ( ) invents the modern concept of the compiler.
1952
The EDVAC is finally completed.
The IBM "Defense Calculator", later renamed the "701", enters production.
Grace Murray Hopper implements the first compiler, the "A-0".

14. In the Beginning – Computing in 1945
Context Computing
Picture from
Harvard Mark I
55 feet long, 8 feet high, 5 tons
Jason Hong / James Landay, UC Berkeley, Picture from

15. Context - Computing in 1945 Ballistics calculations
Physical switches (before microprocessor)
Paper tape
Simple arithmetic & fixed calculations (before programs)
3 seconds to multiply
Picture from
Jason Hong / James Landay, UC Berkeley, /

16. Context - Computing in 1945 First computer bug (Harvard Mark II)
Adm. Grace Murray Hopper
Cobol
Bug & Debug
On the 9th of September, 1947, when the machine was experiencing problems, an investigation showed that there was a moth trapped between the points of Relay #70, in Panel F.
Jason Hong / James Landay, UC Berkeley

17. Innovator: Vannevar Bush
“As We May Think” Atlantic Monthly
“…publication has been extended far beyond our
present ability to make real use of the record.”
Postulated Memex device
Stores all records/articles/communications
Items retrieved by indexing, keywords, cross
references (now called hyperlinks)
Interactive and nonlinear components are key

18. More About Vannevar Bush
Name rhymes with "Beaver"
MIT faculty member
Coordinated WWII effort with 6000 US scientists
Social contract for science
federal government funds universities
universities do basic research
research helps economy & national defense
6000 US scientists
Picture of Bush from

19. Futuristic inventions / trends
“As We May Think”
Futuristic inventions / trends
Wearable cameras for photographic records
Encyclopedia Britanica for a nickel
Automatic transcripts of speech
Memex, Trails of discovery
Direct capture of nerve impulses
Picture from

20. As We May Think Computers don't have to be desktop machines!
The top of the desk would have slanting translucent screens on which material could be projected for convenient reading. The top of the memex would have a transparent platen. When a longhand note, photograph, memoranda, or other things were placed on the platen, the depression of a lever would cause the item to be photographed onto the next blank space in a section of the memex film.
The memex would become "'a sort of mechanized private file and library'.-Bush. It would use microfilm storage, dry photography, and analog computing to give postwar scholars access to a huge, indexed repository of knowledge-any section of which could be called up with a few keystrokes.
Picture from

21. As We May Think Very optimistic about future
Technology could help society
Technology could manage flood of info
Bush – one of most informed people of his time
Look at trends, guess where we're going
If you read it
Which feature is your favorite? Why?
Which feature is your least favorite? Why?
What was he right about? Wrong about?

22. As We May Think Some have come true
Increased specialization
Flood of information
Faster / Cheaper / Smaller / More reliable
Some he missed or we are still waiting
Microphotography?
Digital technologies?
Non-science / Non-office apps?
Memex?

23. As We May Think
Not so much predicting future as "inventing it" by publishing article
hypertext
wearable memory aid
Use technology to augment human intellectual abilities
New kinds of technology lead to new kinds of human/machine & human/human interaction
Be aware that science/engineering can impact society
Picture from

24. Context - Computing in 1960s
Transistor (1948)
ARPA (1958)
Timesharing (1950s)
Terminals and keyboards
Computers still primarily for scientists and engineers
Transistor is better version of vacuum tubes
Modern Pentium III has about 10 million transistors about size of fingernail!
ARPA (Advanced Research Projects Agency) founded to do advanced research, usually military purposes
Founded right after Sputnik
ARPA budget back then only a few million
Modern DARPA budget about 2 billion (2001)
Modern NSF budget about 4.5 billion (2001)
Timesharing let computers be shared among people
Instead of one computer and batch processes, one computer which pretended to be multiple computers
Vacuum Tube
Jason Hong / James Landay, UC Berkeley

25. Batch Processing Computer had one task, performed sequentially
No “interaction” between operator and computer after starting the run
Punch cards, tapes for input
Serial operations

26. Innovator: J. R. Licklider
Postulated “man-computer symbiosis”
Couple human brains and computing machines tightly to revolutionize information handling
“The hope is that, in not too many years, human brains and computing machines will be coupled together very tightly and that the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information-handling machines we know today.”

27. Produced goals that are pre-requisite to “man-computer symbiosis”
Immediate goals:
time sharing of computers among many users
electronic i/o for the display and communication of symbolic and pictorial information
interactive real time system for information processing and programming
large scale information storage and retrieval

28. intermediate goals: long term visions:
facilitation of human cooperation in the design & programming of large systems
combined speech recognition, hand-printed character recognition & light-pen editing
long term visions:
natural language understanding (syntax, semantics, pragmatics)
speech recognition of arbitrary computer users
heuristic programming

29. Need for HCI Mid 1960’s Timesharing mode of computing
Computers too expensive for individuals timesharing increased accessibility
interactive systems, not jobs
text processing, editing
, shared file system
Need for HCI

30. Innovator: Sutherland

31. Sketchpad MIT, 1963 Ivan Sutherland
an innovative program that influenced alternative forms of interaction with computers. .
Sketchpad could accept constraints and specified relationships among segments and arcs, including the diameter of arcs. It could draw both horizontal and vertical lines and combine them into figures and shapes. Figures could be copied, moved, rotated, or resized, retaining their basic properties. Sketchpad also had the first window-drawing program and clipping algorithm, which allowed zooming.

32. Innovator: Ivan Sutherland
SketchPad PhD thesis at MIT
hierarchical structures defined pictures and sub-pictures
object-oriented programming: master picture with instances
constraints: specify details which the system maintains through changes
icons: small pictures that represented more complex items
copying: both pictures and constraints
input techniques: efficient use of light pen
world coordinates: separation of screen from drawing coordinates
recursive operations: applied to children of hierarchical objects
Sophisticated drawing package

33. History of HCI Ivan Sutherland’s SketchPad-1963 PhD
Parallel developments in hardware:
“low-cost” graphics terminals
input devices such as data tablets (1964)
display processors capable of real-time manipulation of images (1968)

34. Technological Advance: Interactive Graphics
More suitable medium than paper - picture worth a thousand words
Start of Direct Manipulation
Computers used for visualizing and manipulating data

36. Bert Sutherland on TX-2

37. MIT Lincoln Lab, Lexington, MA
Lincoln TX-2
MIT Lincoln Lab, Lexington, MA

38. MIT Lincoln Lab, Lexington, MA
Lincoln TX-2
MIT Lincoln Lab, Lexington, MA

39. First VR Head Display
The Sword of Damocles

40. Henri Gouraud: Gouraud shading technique
In Utah
Alan Kay: Smalltalk
Henri Gouraud: Gouraud shading technique
Frank Crow: anti-aliasing methods
1968 University of Utah, Full professor
Smalltalk has influenced the wider world of computer programming in four main areas. It inspired the syntax and semantics of other computer programming languages. Secondly, it was a prototype for a model of computation known as message passing. Thirdly, its WIMP GUI inspired the windowing environments of personal computers in the late twentieth and early twenty-first centuries
first Macintosh desktop look almost identical to the MVC windows of Smalltalk-80. Finally, the integrated development environment was the model for a generation of visual programming tools that look like Smalltalk's code browsers and debuggers.
Python and Ruby have reimplemented some Smalltalk ideas in an environment similar to that of AWK or Perl.

41. In Utah Edwin Catmull: John Warnock: Adobe Jim Clark: Silicon Graphics
co-founder of Pixar
President of Walt Disney and Pixar Animation Studios.
John Warnock: Adobe
Jim Clark: Silicon Graphics
Fletcher Jones Professor at CalTech , Sun

42. Bert Sutherland Sun Microsystems Laboratories
Systems Science Laboratory at Xerox PARC
Computer Science Division of Bolt, Beranek and Newman, Inc: ARPANET
Claude Shannon

43. The Ubiquitous ASR 33 Teletype
ASR: Automatic Send / Receive
Save programs on punched paper tape
The first direct human-computer interface experience for many in the 1960s
About 10 characters per second bps

44. The Ubiquitous Glass Teletype
24 x 80 characters
Up to 19,200 bps (Wow - was big stuff!)
Construction: Monitor + detachable keyboard
Display: x80 or 14x132 character cells, optional 24x132
Character matrix: x9 with descenders
Screen size: " diagonal (8" x 4.5" active display)
Character set: Complete US ASCII (128 codes)
Keys: keys in ANSI X typewriter layout
Auxiliary keypad: keys (digits, arrows, function keys)
Visual indicators: LEDs
Interface: RS-232/V.24, optional 20mA Current Loop
Flow control: Xon/Xoff
Communication Speeds: 75,110,150,300,600,1200,2400,4800,9600,19200 bps
Dimensions: "x18"14.25" (monitor), 3.5"x18"x8" (keyboard)
Minimum table depth: "
Weight: pounds
Source:

45. Technological Advance / Paradigm Shift: Time Sharing
(Mid 1960s)
Command line - teletypes, then “glass teletypes”
Computers still too expensive for individuals timesharing
increased accessibility
interactive systems, not jobs
text processing, editing
, shared file system
* There was an unrecognized need for HCI in the design of programming languages
Need
for
HCI*
Need to add a picture of a teletype and glass teletype

46. Innovator: Douglas Englebart
Landmark system/demo:
Hierarchical hypertext, multimedia, mouse, high-res display, windows, shared files, electronic messaging, CSCW, teleconferencing, ...
Invented the mouse

47. The First Mouse (1964)
Over the next 6 years, he created a team, acquired funding, and started implementing his vision. Along the way, he and
his group invented the mouse, but as we will see this is
only a small part of the complete project.

48. AFIP Fall Joint Conference, 1968
Document Processing
modern word processing
outline processing
hypermedia
Input / Output
the mouse and one-handed corded keyboard
high resolution displays
multiple windows
specially designed furniture
Shared work
shared files and personal annotations
electronic messaging
shared displays with multiple pointers
audio/video conferencing
ideas of an Internet
User testing, training
This phase of his work peaked in 1968, where he took a huge risk by deciding to present his ideas and the developing
NLS system in a live demonstration at the AFIP Fall Joint Computer Conference.
Many concepts in today's interfaces were first introduced in the NLS
system and its presentation
Document Processing
modern word processing
outline processing
hypermedia

49. About Doug Engelbart Graduate of Berkeley (EE '55)
"bi-stable gaseous plasma digital devices"
Stanford Research Institute (SRI)
Augmentation Research Center
1962 Paper "Conceptual Model for Augmenting Human Intellect"
Complexity of problems increasing
Need better ways of solving problems
Picture of Engelbart from bootstrap.org
Picture from

50. Augmenting Human Intellect
Advantages of chorded keyboards?
Disadvantages?
Jason Hong / James Landay, UC Berkeley,

51. Augmenting Human Intellect
“At SRI in the 1960s we did some experimenting with a foot mouse. I found that it was workable, but my control wasn't very fine and my leg tended to cramp from the unusual posture and task.”

52. Augmenting Human Intellect
Early
3-button
mouse
Chorded
Keyboard

53. Augmenting Human Intellect
First mouse
First hypertext
First word processing
First 2D editing and windows
First document version control
First groupware (shared screen teleconferencing)
First context-sensitive help
First distributed client-server
Many, many more!

54. Augmentation not Automation
"I tell people: look, you can spend all you want on building smart agents and smart tools…" "I'd bet that if you then give those to twenty people with no special training, and if you let me take twenty people and really condition and train them especially to learn how to harness the tools…" "The people with the training will always outdo the people for whom the computers were supposed to do the work."
Not AI, co-evolution of people and tools

55. Augmenting Human Intellect
Example: Roman Numerals vs Arabic
What is XCI + III?
Now what is XCI x III?
What is 91 * 3?
New kinds of artifacts, languages, methodologies, and training can enable us to do things we couldn't do before or simplify what we already do

56. Computers as Toolkits
Multipurpose toolkits
Abstracting out common tasks (tools)
Reusable elements
At the disposal of humans

57. Paradigm Shifter: Alan Kay
“Personal Computing”
Dynabook: Notebook sized computer loaded with multimedia and can store everything
Desktop interface metaphor

58. Paradigm Shifter: Ted Nelson
Computers can help people, not just business
Coined term “hypertext”

59. Personal Computers 1974 IBM 5100 1981 Datamaster 1981 IBM XT/AT
Text and command-based
Sold lots
Performed lots of tasks the general public wanted done
A good basic toolkit
1978 VisiCalc

60. Personal Computing System is more powerful if it’s easier to use
Small, powerful machines dedicated to individual
Importance of networks and time-sharing
Kay’s Dynabook, IBM PC
Time names “The Computer” Man of the Year, 1982 (

61. WIMP Windows, Icons, Menus, Pointers
Timesharing=multiusers; now we need multitasking
WIMP interface allows you to do several things simultaneously
Has become the familiar GUI interface
Xerox Alto, Star; early Apples

62. PCs with GUIs Xerox PARC - mid 1970’s Alto
Alto computer, a personal workstation
local processor, bit-mapped display, mouse
modern graphical interfaces
text and drawing editing, electronic mail
windows, menus, scroll bars, mouse selection, etc
local area networks (Ethernet) for personal workstations
could make use of shared resources

63. Commercial machines: Xerox Star - 1981
First commercial PC designed for “business professionals”
desktop metaphor, pointing, WYSIWYG, consistency and simplicity

64. Xerox Star-1981
First commercial personal computer designed for “business professionals”
First comprehensive GUI used many ideas developed at Xerox PARC
familiar user’s conceptual model (simulated desktop)
promoted recognizing/pointing rather than remembering/typing
property sheets to specify appearance/behaviour of objects
what you see is what you get (WYSIWYG)
small set of generic commands that could be used throughout the system
high degree of consistency and simplicity
modeless interaction
limited amount of user tailorability

65. Xerox Star (continued)
First system based upon usability engineering
inspired design
extensive paper prototyping and usage analysis
usability testing with potential users
iterative refinement of interface
Commercial failure
cost ($15,000);
IBM had just announced a less expensive machine
limited functionality
e.g., no spreadsheet
closed architecture,
3rd party vendors could not add applications
perceived as slow
but really fast!
slavish adherence to direct manipulation

66. Apple Lisa - ‘82 Based on ideas of Star
More personal rather than office tool
Still $$$
Failure (why?)
Too slow, too expensive, MacIntosh came out at better price point as well as IBM PC

67. Apple Macintosh - ‘84 “Old ideas” but well done!
Aggressive pricing - $2500
Not trailblazer, smart copier
Good interface guidelines
3rd party applications
High quality graphics and laser printer

68. Commercial Machines: Apple Macintosh (1984)
Succeeded because:
aggressive pricing ($2500)
did not need to trailblaze
learnt from mistakes of Lisa and corrected them; ideas now “mature”
market now ready for them
developer’s toolkit encouraged 3rd party non-Apple software
interface guidelines encouraged consistency between applications
domination in desktop publishing because of affordable laser printer and excellent graphics

69. Direct Manipulation
‘82 Shneiderman describes appeal of graphically-based interaction
object visibility
incremental action and rapid feedback
reversibility encourages exploration
replace language with action
syntactic correctness of all actions
WYSIWYG, Apple Mac

70. Other events: MIT Architecture Machine Group ACM SIGCHI (1982)
Nicholas Negroponte ( )
many innovative inventions, including
wall sized displays
use of video disks
use of artificial intelligence in interfaces (idea of agents)
speech recognition merged with pointing
speech production
multimedia hypertext
....
ACM SIGCHI (1982)
special interest group on computer-human interaction
conferences draw between people
HCI Journals
Int J Man Machine Studies (1969)
many others since 1982

71. HCI importance result of:
cheaper/available computers/workstations meant people more important than machines
excellent interface ideas modeled after human needs instead of system needs (user centered design)
evolution of ideas into products through several generations
pioneer systems developed innovative designs, but often commercially unviable
settler systems incorporated (many years later) well-researched designs
people no longer willing to accept products with poor interfaces

72. Metaphor All use is problem-solving or learning to some extent
Relating computing to real-world activity is effective learning mechanism
File management on office desktop
Financial analysis as spreadsheets

73. Speech, Language? Actions do not always speak louder than words
Interface as mediator or agent
Language paradigm
How good does it need to be?
“Tricks”, vocabulary, domains
How “human” do we want it to be?

74. Hypertext
Think of information not as linear flow but as interconnected nodes
Nelson’s hypertext
Bush’s MEMEX
Non-linear browsing
WWW ‘93
Hypermedia

75. The Interconnected Web
The Network is the Computer
e.g.

78. ?
Thanks