1. Computer Graphics - Input & Interaction-
Hanyang University
Jong-Il Park

2. Objectives Introduce the basic input devices Event-driven input
Physical Devices
Logical Devices
Input Modes
Event-driven input
Double buffering for smooth animations
Programming event input with GLUT
Virtual reality: interaction + reality

3. Project Sketchpad
Ivan Sutherland (MIT 1963) established the basic interactive paradigm that characterizes interactive computer graphics:
User sees an object on the display
User points to (picks) the object with an input device (light pen, mouse, trackball)
Object changes (moves, rotates, morphs)
Repeat

4. Graphical Input Devices can be described either by Modes
Physical properties
Mouse
Keyboard
Trackball
Logical properties
What is returned to program via API
A position
An object identifier
Modes
How and when input is obtained
Request or event

5. Physical Devices mouse trackball light pen data tablet joy stick
space ball

6. Incremental (Relative) Devices
Devices such as the data tablet return a position directly to the operating system
Devices such as the mouse, trackball, and joy stick return incremental inputs (or velocities) to the operating system
Must integrate these inputs to obtain an absolute position
Rotation of cylinders in mouse
Roll of trackball
Difficult to obtain absolute position
Can get variable sensitivity

7. Logical Devices Consider the C and C++ code What is the input device?
C++: cin >> x;
C: scanf (“%d”, &x);
What is the input device?
Can’t tell from the code
Could be keyboard, file, output from another program
The code provides logical input
A number (an int) is returned to the program regardless of the physical device

8. Input Modes
Input devices contain a trigger which can be used to send a signal to the operating system
Button on mouse
Pressing or releasing a key
When triggered, input devices return information (their measure) to the system
Mouse returns position information
Keyboard returns ASCII code

9. Request Mode
Input provided to program only when user triggers the device
Typical of keyboard input
Can erase (backspace), edit, correct until enter (return) key (the trigger) is depressed
Eg. scanf()

10. Event Mode
Most systems have more than one input device, each of which can be triggered at an arbitrary time by a user
Each trigger generates an event whose measure is put in an event queue which can be examined by the user program

11. Event Types Window: resize, expose, iconify
Mouse: click one or more buttons
Motion: move mouse
Keyboard: press or release a key
Idle: nonevent
Define what should be done if no other event is in queue

12. Callbacks Programming interface for event-driven input
Define a callback function for each type of event the graphics system recognizes
This user-supplied function is executed when the event occurs
GLUT example: glutMouseFunc(mymouse)

13. GLUT callbacks
GLUT recognizes a subset of the events recognized by any particular window system (Windows, X, Macintosh)
glutDisplayFunc
glutMouseFunc
glutReshapeFunc
glutKeyboardFunc
glutIdleFunc
glutMotionFunc, glutPassiveMotionFunc

14. GLUT Event Loop
Recall that the last line in main.c for a program using GLUT must be
glutMainLoop();
which puts the program in an infinite event loop
In each pass through the event loop, GLUT
looks at the events in the queue
for each event in the queue, GLUT executes the appropriate callback function if one is defined
if no callback is defined for the event, the event is ignored

15. The display callback
The display callback is executed whenever GLUT determines that the window should be refreshed, for example
When the window is first opened
When the window is reshaped
When a window is exposed
When the user program decides it wants to change the display
In main.c
glutDisplayFunc(mydisplay) identifies the function to be executed
Every GLUT program must have a display callback

16. Posting redisplays
Many events may invoke the display callback function
Can lead to multiple executions of the display callback on a single pass through the event loop
We can avoid this problem by instead using
glutPostRedisplay();
which sets a flag.
GLUT checks to see if the flag is set at the end of the event loop
If set then the display callback function is executed

17. Animating a Display
When we redraw the display through the display callback, we usually start by clearing the window
glClear()
then draw the altered display
Problem: the drawing of information in the frame buffer is decoupled from the display of its contents
Graphics systems use dual ported memory
Hence we can see partially drawn display
See the program single_double.c for an example with a rotating cube

18. Double Buffering Instead of one color buffer, we use two
Front Buffer: one that is displayed but not written to
Back Buffer: one that is written to but not displayed
Program then requests a double buffer in main.c
glutInitDisplayMode(GL_RGB | GL_DOUBLE)
At the end of the display callback buffers are swapped
void mydisplay() {
glClear(GL_COLOR_BUFFER_BIT|….) .
/* draw graphics here */
glutSwapBuffers() }

19. Using the idle callback
The idle callback is executed whenever there are no events in the event queue
glutIdleFunc(myidle)
Useful for animations
void myidle() {
/* change something */
t += dt
glutPostRedisplay(); }
Void mydisplay() {
glClear();
/* draw something that depends on t */
glutSwapBuffers();

20. Using globals The form of all GLUT callbacks is fixed
void mydisplay()
void mymouse(GLint button, GLint state, GLint x, GLint y)
Must use globals to pass information to callbacks
float t; /*global */
void mydisplay() {
/* draw something that depends on t }

21. Virtual Reality?
Head-mounted Display (HMD)

22. Virtual Reality?
BOOM (Binocular Omni-Orientation Monitor)

23. Virtual Reality?
CAVE (Cave Automatic Virtual Environment)

24. Virtual Reality?
Shared Virtual Environment

25. Virtual Reality Virtual 실제로는 존재하지 않지만, 본질적으로 존재하는 것과 동등한 효과를 가지는 것
실제 일어난 일, 현실감
Virtual Reality
실제로는 존재하지 않지만, 실제로 존재하는 것과 동등한 사실이나 사건

26. Def. VR What is Virtual Reality?
“A high-end user interface that involves real-time simulation and interaction through multiple sensorial channels.” (vision, sound, touch, smell, taste)

27. VR의 3 요소 Presence Interaction Autonomy 사실적 reality, 몰입적 reality
by D.Zelter (1992)

28. Three “I”s of VR
Burdea, 1993.
Immersion I 3
Interaction
Imagination

29. Similar concepts (I) Virtual reality (1989, J.Lanier, VPL)
“계산기에 의해 합성된 인공적 세계”
Artificial reality(Mid-’70, M.Krueger)
VIDEOPLACE: 실제처럼 느끼면서 컴퓨터가 만든 세계에 참여하여 상호작용하는 것
Cyberspace(1983, William Gibson)
“a single artificial reality that could be experienced simultaneously by thousands of people worldwide”

30. Similar concepts (II) Telepresence or Tele-existence
우주공간, 심해 등 그대로는 체험하기 불가능한 곳의 상황을 현실감 넘치게 제시
현실의 복제라는 점에서 VR과 차별화됨
Augmented reality - virtual objects in real world
Augmented virtuality - real objects in virtual world
Mixed reality
기타: synthetic reality, virtual environment, virtual world, artificial life

31. Augmented Reality
(ARTEMIS, U.Toronto)

32. Augmented Virtuality (

33. MR Includes… AV
real
virtual AR

34. Reality vs. Virtuality Reality = Unmodelled Virtuality = Modelled
Reality-Virtuality Continuum
Real
Environment
Virtual
Environment
Extent of World Knowledge Continuum
World
Unmodelled
World
Completely
Modelled

35. History of VR(I) Sensorama Simulator (Morton Heilig, 1962)
head-mounted television( “ , 1960)
Head-Mounted Display(Ivan Sutherland, 1966)
Scene generator(Evans and Sutherland)
simple scene about polygons at 20 scenes/sec (1973)
METAPLAY(1970) ,VIDEOPLACE(1974, Myron Krueger)
Flight simulators etc. by military(70s to early 80s)

36. Sensorama Simulator
Heilig, US Patent #3,050,870, 1962

37. Ivan Sutherland’s HMD (1966+)

38. Videoplace by M.Krueger

39. History of VR(II)
VIVED(Virtual Visual Environment Display) project (NASA)
LCD(Liquid Crystal Display)-based HMD (1981)
Polhemus noncontact tracker – user’s head tracking
Sensing glove (Fisher, 1985)
VIVED -> VIEW(Virtual Interface Environment Workstation)
3D virtual sound source (Fisher and Wenzel, 1988)
image wireframe rendering with flat shaded surface

40. History of VR(III)
VPL, 1st commercial company selling VR products (1987)
DataGloves
EyePhones - 1st HMD
“Vision”, 1st integrated commercial VR workstation (Division Ltd., UK, 1991)
“WorldToolKit”, 1st VR development software
(Sense8, 1992)
Mixed Reality (ATR, 1994)
Plenoptic modeling and image-based rendering (McMillan and Bishop, 1995)
Virtualized Reality (CMU, 1996)

41. The VPL DataGlove (1987)

42. NASA VIEW system (1989)

43. History of VR(IV) VRML(Virtual Reality Modeling Language)
VRML 2.0 (1996), VRML97: dynamic scene animation
Currently
Java 3D API, VRTP(VR Transfer Protocol) …
Mixed reality applications

44. Recent Technologies in VR (I)
Computer vision and computer graphics
Image-based rendering
Large scale visualization
mosaic
Robust geometric registration
Motion/depth-keying
Occlusion
Image segmentation

45. Recent Technologies in VR (II)
Devices
See-through HMD (ST-HMD)
Optical, Video ST-HMD
Wearable equipments
New tracking sensors for outdoor

46. Recent Technologies in VR (III)
New interfaces
Gesture recognition
Tangible interface
Haptic devices
Hybrid interfaces

47. Applications of VR (I) Mobile Collaborative Commercial
Outdoor navigation with wearable equipments
Military training, sightseeing
Collaborative
Simultaneous and interactive multi-users
Shared virtual environment
Commercial
Broadcasting: character service
Entertainment
Film industry

48. Applications of VR (II)
Various simulator
Flight, car driving
Military training
Building, factory, City modeling
Medical application
Human body 3D modeling (CT, MR)
Tele-surgery
Limitations of future VR
Technology, interface, social acceptance

49. Relation of VR, CG, and CV VR

50. Major Issues in MR
Accurate registration between virtual and real world
Realistic display(vision, sound, touch, smell, taste)
All should be in REAL-TIME
Analysis Time(TA)
Synthesis Time(TS)
Feedback Time(= TA + TS)
Should be small enough
Should not annoy human’s feeling of reality

51. Critical Enabling Technology
Accurate and fast registration between RW and VW !!!
In more detail, a good tracker
Fast
Light and small
Low power-consumption
Accurate
Immune to interference

52. ? Where Are We Heading? Art Psychology Human science CG/CV Computers
Mechanics
Sensors
HCI