1. CS 638, Fall 2001 Interactive Programs Games are interactive systems - they must respond to the user Today is all about how interactive programs are designed Also, some issues to think about in interactive systems

2. CS 638, Fall 2001 Interactive Program Structure Event driven programming –Everything happens in response to an event Events come from two sources: –The user –The system Events are also called messages –An event causes a message to be sent… Initialize User Does Something or Timer Goes Off System Updates

3. CS 638, Fall 2001 User Events The OS manages user input –Interrupts at the hardware level … –Get converted into events in queues at the windowing level … –Are made available to your program It is generally up to the application to make use of the event stream Windowing systems may abstract the events for you

4. CS 638, Fall 2001 System Events Windowing systems provide timer events –The application requests an event at a future time –The system will provide an event sometime around the requested time. Semantics vary: Guaranteed to come before the requested time As soon as possible after Almost never right on (real-time OS?) Application is not interrupted - it has to look for the event –Exception: Alarm signals in UNIX

5. CS 638, Fall 2001 Polling for Events Most windowing systems provide a non-blocking event function –Does not wait for an event, just returns NULL if one is not ready What type of games might use this structure? Why wouldn’t you always use it? while ( true ) if ( e = checkEvent() ) switch ( e.type ) … do more work

6. CS 638, Fall 2001 Waiting for Events Most windowing systems provide a blocking event function –Waits (blocks) until an event is available –Usually used with timer events. Why? On what systems is this better than the previous method? What types of games is it useful for? e = nextEvent(); switch ( e.type ) …

7. CS 638, Fall 2001 The Callback Abstraction A common event abstraction is the callback mechanism Applications register functions they wish to have called in response to particular events –Translation table says which callbacks go with which events Generally found in GUI (graphical user interface) toolkits –“When the button is pressed, invoke the callback” –FLTK works this way –Many systems mix methods, or have a catch-all callback for unclaimed events –In X windows, Xlib defines events, Xtoolkit defines callbacks Why are callbacks good? Why are they bad?

8. CS 638, Fall 2001 LithTech Messaging LithTech allows any of the methods we have discussed –Callbacks for the system timer the update function called once per frame –Callbacks bound to multiple events Same callback gets many events and internally decides what to do with each one Bindings done with a configuration file –Polling for keyboard input. What sort of input? Read the Programming Guide chapter on Input to get an overview

9. CS 638, Fall 2001 Upon Receiving an Event … Event responses fall into two classes: –Task events: The event sparks a specific task or results in some change of state within the current mode eg Load, Save, Pick up a weapon, turn on the lights, … Call a function to do the job –Mode switches: The event causes the game to shift to some other mode of operation eg Start game, quit, go to menu, … Switch event loops, because events now have different meanings Software structure reflects this - menu system is separate from run-time game system, for example

10. CS 638, Fall 2001 Real-Time Loop At the core of games with animation is a real-time loop: What else might you need to do? The number of times this loop executes per second is the frame rate –# frames per second (fps) while ( true ) process events update animation render

11. CS 638, Fall 2001 Lag Lag is the time between when a user does something and when they see the result - also called latency –Too much lag and causality is distorted –With tight visual/motion coupling, too much lag makes people motion sick Big problem with head-mounted displays for virtual reality –Too much lag makes it hard to target objects (and track them, and do all sorts of other perceptual tasks) High variance in lag also makes interaction difficult –Users can adjust to constant lag, but not variable lag From a psychological perspective, lag is the important variable

12. CS 638, Fall 2001 Computing Lag Lag is NOT the time it takes to compute 1 frame! What is the formula for maximum lag as a function of frame rate, fr? What is the formula for average lag? Process input Update state Render Process input Update state Render Process input Frame time Lag Event time

13. CS 638, Fall 2001 Frame Rate Questions What is an acceptable frame rate for twitch games? Why? What is the maximum useful frame rate? Why? What is the frame rate for NTSC television? What is the minimum frame rate required for a sense of presence? How do we know? How can we manipulate the frame rate?

14. CS 638, Fall 2001 Frame Rate Answers (I) Twitch games demand at least 30fs, but the higher the better (lower lag) –Users see enemy’s motions sooner –Higher frame rates make targeting easier The maximum useful frame rate is the monitor refresh rate –Time taken for the monitor to draw one screen –Synchronization issues Buffer swap in graphics is timed with vertical sweep, so ideal frame rate is monitor refresh rate Can turn of synchronization, but get nasty artifacts on screen

15. CS 638, Fall 2001 Frame Rate Answers (II) NTSC television draws all the odd lines of the screen, then all the even ones (interlace format) –Full screen takes 1/30th of a second –Use 60fps to improve visuals, but only half of each frame actually gets drawn by the screen –Do consoles only render 1/2 screen each time? It was once argued that 10fps was required for a sense of presence (being there) –Head mounted displays require 20fps or higher to avoid illness –Many factors influence the sense of presence –Perceptual studies indicate what frame rates are acceptable

16. CS 638, Fall 2001 Reducing Lag Faster algorithms and hardware is the obvious answer Designers choose a frame rate and put as much into the game as they can without going below the threshold –Part of design documents presented to the publisher –Threshold assumes fastest hardware and all game features turned on –Options given to players to reduce game features and improve their frame rate There is a resource budget: How much of the loop is dedicated to each aspect of the game (graphics, AI, sound, …) Some other techniques allow for more features and less lag

17. CS 638, Fall 2001 Decoupling Computation It is most important to minimize lag between the user actions and their direct consequences –So the input/rendering loop must have low latency Lag between actions and other consequences may be less severe –Time between input and the reaction of enemy can be greater –Time to switch animations can be greater Technique: Update different parts of the game at different rates, which requires decoupling them –For example, run graphics at 60fps, AI at 10fps –Done in Unreal engine, for instance

18. CS 638, Fall 2001 Animation and Sound Animation and sound need not be changed at high frequency, but they must be updated at high frequency –For example, switching from walk to run can happen at low frequency, but joint angles for walking must be updated at every frame Solution is to package multiple frames of animation and submit them all at once to the renderer –Good idea anyway, makes animation independent of frame rate Sound is offloaded to the sound card

19. CS 638, Fall 2001 Graphics Review Recall the standard graphics pipeline:

20. CS 638, Fall 2001 Normal Vectors The intensity of a surface depends on its orientation with respect to the light and the viewer –CDs are an extreme example The surface normal vector describes the orientation of the surface at a point –Mathematically: Vector that is perpendicular to the tangent plane of the surface What’s the problem with this definition? –Just “the normal vector” or “the normal” –Will use N to denote

21. CS 638, Fall 2001 Local Shading Models Local shading models provide a way to determine the intensity and color of a point on a surface –The models are local because they don’t consider other objects at all –We use them because they are fast and simple to compute –They do not require knowledge of the entire scene, only the current piece of surface

22. CS 638, Fall 2001 Local Shading Models (Watt 6.2) What they capture: –Direct illumination from light sources –Diffuse and Specular components –(Very) Approximate effects of global lighting What they don’t do: –Shadows –Mirrors –Refraction –Lots of other stuff …

23. CS 638, Fall 2001 “Standard” Lighting Model Consists of three terms linearly combined: –Diffuse component for the amount of incoming light reflected equally in all directions –Specular component for the amount of light reflected in a mirror-like fashion –Ambient term to approximate light arriving via other surfaces

24. CS 638, Fall 2001 Diffuse Illumination Incoming light, I i, from direction L, is reflected equally in all directions –No dependence on viewing direction Amount of light reflected depends on: –Angle of surface with respect to light source Actually, determines how much light is collected by the surface, to then be reflected –Diffuse reflectance coefficient of the surface, k d Don’t want to illuminate back side. Use

25. CS 638, Fall 2001 Diffuse Example Where is the light?

26. CS 638, Fall 2001 Specular Reflection (Phong Model) Incoming light is reflected primarily in the mirror direction, R –Perceived intensity depends on the relationship between the viewing direction, V, and the mirror direction –Bright spot is called a specularity Intensity controlled by: –The specular reflectance coefficient, k s –The parameter, n, controls the apparent size of the specularity Higher n, smaller highlight L R V

27. CS 638, Fall 2001 Specular Example

28. CS 638, Fall 2001 Putting It Together Global ambient intensity, I a : –Gross approximation to light bouncing around of all other surfaces –Modulated by ambient reflectance k a Just sum all the terms If there are multiple lights, sum contributions from each light Several variations, and approximations …

29. CS 638, Fall 2001 Flat shading Compute shading at a representative point and apply to whole polygon –OpenGL uses one of the vertices Advantages: –Fast - one shading value per polygon Disadvantages: –Inaccurate –Discontinuities at polygon boundaries

30. CS 638, Fall 2001 Gourand Shading Shade each vertex with it’s own location and normal Linearly interpolate across the face Advantages: –Fast - incremental calculations when rasterizing –Much smoother - use one normal per shared vertex to get continuity between faces Disadvantages: –Specularities get lost

31. CS 638, Fall 2001 Phong Interpolation Interpolate normals across faces Shade each pixel Advantages: –High quality, narrow specularities Disadvantages: –Expensive –Still an approximation for most surfaces Not to be confused with Phong’s specularity model

32. CS 638, Fall 2001