2. Further games and graphics concepts COSE50581 Introduction to Module and Recap Bob Hobbs Faculty of Computing, Engineering and Technology Staffordshire University

3. Outline Module Details Module Details What is 3D programming? What is 3D programming? Physics & AI Physics & AI Typical Processing Steps Typical Processing Steps Modelling and Rendering Modelling and Rendering Windows and Direct X Windows and Direct X Programmable Pipeline Programmable Pipeline Summary Summary

4. Module Details Teaching Team Teaching Team –Bob Hobbsr.g.hobbs@staffs.ac.uk r.g.hobbs@staffs.ac.uk –Steve Foster s.foster@staffs.ac.uk s.foster@staffs.ac.uk –Carl Mitchell c.mitchell@staffs.ac.uk c.mitchell@staffs.ac.uk Semester 1 & 230 cats Semester 1 & 230 cats 3 Hours per week 3 Hours per week –Weeks 1 - 6 (2hrs Lecture + 1hr Tutorial) –Weeks 7 – 12 (1hr Lecture + 2hrs Tutorial) –Timetable Timetable

5. Module Details Course Handbook & Lecture Notes Specification Specification Specification Blackboard (not yet live) Blackboard (not yet live) Assignment Details 70% assignment work 70% assignment work –Phased submission with reflective report components 30% two hour exam at end of year 30% two hour exam at end of year

6. Program of Study – Semester 1 Week 01 Introduction to Module and DirectX11, Maths Week 01 Introduction to Module and DirectX11, Maths Week 02 Geometry, Transformations, Colour using the programmable pipeline Week 02 Geometry, Transformations, Colour using the programmable pipeline Week 03 Depth, Stencil Buffer, Simple Lighting Week 03 Depth, Stencil Buffer, Simple Lighting Week 04Camera, Transformations Week 04Camera, Transformations Week 05Data structures for graphics engines Week 05Data structures for graphics engines Week 06 Advanced Lighting and Shading Week 06 Advanced Lighting and Shading Week 07Textures & Texture Mapping, Render States Week 07Textures & Texture Mapping, Render States Week 08Cube, Normal and Bump mapping Week 08Cube, Normal and Bump mapping Week 09Meshing and Instancing Week 09Meshing and Instancing Week 10 Culling, Clipping, Depth and Line Drawing Week 10 Culling, Clipping, Depth and Line Drawing Week 11Tessellation and Terrain Week 11Tessellation and Terrain Week 12Review and Assessment Week 12Review and Assessment

7. Program of Study – Semester 2 Week 01 Animation and Timing Week 01 Animation and Timing Week 02 Newtonian Physics (velocity, motion, forces) Week 02 Newtonian Physics (velocity, motion, forces) Week 03 Gravity and friction Week 03 Gravity and friction Week 04Particles and Ballistics Week 04Particles and Ballistics Week 05Rotational Physics Week 05Rotational Physics Week 06 Collision Detection and Response Week 06 Collision Detection and Response Week 07Principles of AI, Behaviours Week 07Principles of AI, Behaviours Week 08Pathfinding and searching, Autonomous motion Week 08Pathfinding and searching, Autonomous motion Week 09Ai for scene optimisation Week 09Ai for scene optimisation Week 10 State management Week 10 State management Week 11Decisions and cognition Week 11Decisions and cognition Week 12Review and Assessment Week 12Review and Assessment

8. What is 3D rendering? Object Viewer Generally deals with graphical display of 3D objects as seen by viewer Synthetic image will vary according to: viewing direction, viewer position, illumination, object properties,... Projection onto 2D surface

9. What is 3D Computer Graphics? 3D graphics: generation of graphical display (rendering) of 3D object(s) from specification (model(s)) 3D graphics: generation of graphical display (rendering) of 3D object(s) from specification (model(s)) Specification Modelling Graphical display Rendering such as

10. Wireframe polygonal modelSolid object Vertices Viewpoint Typical Processing Steps Facets Transformation Hidden surface removal Light source Shading

11. Object model(s) Illumination model Viewing and projection specification Graphics engine Graphical display Rendering: operations that produce view of scene projected onto view surface Modelling: numerical description of scene objects, illumination, and viewer Typical Processing Steps

12. 23-Oct-1511 2D picture Light source Surface material Typical Processing Steps xmxm zmzm ymym xmxm zmzm ymym 3D world coordinates 3D modelling coordinates 2D device coordinates 3D viewing coordinates & Culling xwxw zwzw ywyw xvxv zvzv yvyv Graphics pipeline: generation of picture involves sequence of coordinate transformations, hidden surface removal, shading/texturing and etc...

13. 23-Oct-1512 Typical graphics pipeline Typical Processing Steps Pos., orient. of display area Geometry models Shapes World pos., orient., sizes Viewpoint Pos., orient. Field of view Projection type Viewport Object & back-face culling View mapping onto display device Generation / configuration of world Mesh operations Geometry processing: shape, world geometry, viewpoint 2D picture Lights & materials Shading, texturing,... Light pos., orient. Optical char. Pixel operations Material processing: colour, texture, transparency/occlusion, … Hidden surface removal Viewpoint

14. 23-Oct-1513 Typical Processing Steps Loading or generation of object (shape, surface properties) model Loading or generation of object (shape, surface properties) model Model coordinate transformation: translation, rotation, projection,... Model coordinate transformation: translation, rotation, projection,... Hidden surface removal Hidden surface removal Shading Shading Display Display

15. 23-Oct-1514 Viewport transformation 3D Models Modelling transformation 2D Display Back-face culling & Vertex lighting Viewing transformation Object culling Clipping could also be performed before viewing transformation, or during rasterisation Polygon clipping Typical Processing Steps Projection Scan conv. HSR, Texturing, & Shading Typical graphics pipeline

16. Modelling Human Head Model 1438 facets

17. Modelling Human Head Model 7258 facets

18. Modelling Teacher and Board Model 2074 facets

19. Rendering Shaded Human Head 1438 facets

20. Rendering Shaded Human Head 7258 facets

21. Rendering Shaded Teacher and Board

22. 23-Oct-1521 Triangle scan conversion Rendering requires scan conversion Rendering requires scan conversion Scan conversion of graphics primitive: Scan conversion of graphics primitive: – point-by- point traversal of primitive along (horizontal) scan lines along (horizontal) scan lines coherence often exploited (e.g. incremental computation) coherence often exploited (e.g. incremental computation) å reduces computational cost Rendering

23. 23-Oct-1522 x y Rasterisation of filled triangle Point grid (raster) Scan line Point / Pixel External pointRendering b Common approach: polygon interior filled line by line, one point at a time (rasterisation) linear interpolation (i.e. line equation) often used for colour shade, depth and other calculations at each pixellinear interpolation (i.e. line equation) often used for colour shade, depth and other calculations at each pixel

24. 23-Oct-1523 General procedure for filled triangles: find intersections of scan line with polygon edge find intersections of scan line with polygon edge – use equation of line joining two vertices: interpolation edge coherence, hence use incremental computation edge coherence, hence use incremental computation fill / shade / texture interior points between intersection pairs along scan line fill / shade / texture interior points between intersection pairs along scan line Rendering

25. 23-Oct-1524 General procedure for filled triangles: find intersections of scan line with polygon edge find intersections of scan line with polygon edge – use equation of line joining two vertices: interpolation edge coherence, hence use incremental computation edge coherence, hence use incremental computation fill / shade / texture interior points between intersection pairs along scan line fill / shade / texture interior points between intersection pairs along scan line Rendering

26. Windows Programming Rendering takes place in a window Rendering takes place in a window Application must generate window in which to render graphics Application must generate window in which to render graphics IP3DA used C and GLUT to handle windows. OpenGL can run in GLUT or other windowing framework including MS windows IP3DA used C and GLUT to handle windows. OpenGL can run in GLUT or other windowing framework including MS windows DirectX is integrated into windows programming API DirectX is integrated into windows programming API

27. What is a “window?” The most obvious windows are the application windows on your desktop. The most obvious windows are the application windows on your desktop. The term “window” also refers to any child window or control (such as a button) within a window. The term “window” also refers to any child window or control (such as a button) within a window. Windows are organized in a hierarchy. Windows are organized in a hierarchy.

28. Example - Calculator

29. Event Driven Programming Windows programming is event driven. Windows programming is event driven. An event driven program sits and waits for an event to process. An event driven program sits and waits for an event to process. Events include moving or sizing a window, clicking the mouse button over a window, or typing keys on the keyboard. Events include moving or sizing a window, clicking the mouse button over a window, or typing keys on the keyboard. Windows programs are notified of events by messages sent from the OS. Windows programs are notified of events by messages sent from the OS.

30. Win32 API We will use the Win32 Platform API to write a windows program. We will use the Win32 Platform API to write a windows program. –Typically more tedious, but we only need to create one window. –Faster and Smaller executable. –Written in C++ Other options include: Other options include: –MFC, Qt, wxWindows, SDL,.NET Forms

31. WinMain Instead of main, windows programs use WinMain Instead of main, windows programs use WinMain WinMain has different parameters: WinMain has different parameters: – HINSTANCE hInstance – a handle to the program – HINSTANCE hPrevInstance – no longer used. – LPSTR lpCmdLine – unparsed command line. Doesn’t include executable’s filename. – int nCmdShow – Specifies how the window should be initially shown (ie. Minimized, Maximized, Normal) WinMain must have a WINAPI modifier. WinMain must have a WINAPI modifier. – int WINAPI WinMain (…) WinMain still returns an integer like main. WinMain still returns an integer like main.

32. Headers To use the Win32 API we need to #include windows.h To use the Win32 API we need to #include windows.h If we #define WIN32_LEAN_AND_MEAN before #including windows.h, the compiler will skip compiling the more rarely used windows code. If we #define WIN32_LEAN_AND_MEAN before #including windows.h, the compiler will skip compiling the more rarely used windows code.

33. Windows Program Outline Create Window Class Create Window Class Create Window Create Window Show the Window Show the Window Enter Message Loop Enter Message Loop

34. Window Classes A window class acts as a template for window creation. A window class acts as a template for window creation. Window classes are not classes in the C++ sense. Window classes are not classes in the C++ sense. We can set such properties as: We can set such properties as: –Name of the window class – important for identifying it later! –Window style – How it looks and its default behaviors. –Window procedure – a pointer to function that handles messages for a window. –Default cursor, icon, and menu.

35. Creating a Window Class Fill out a WNDCLASS or WNDCLASSEX structure. Fill out a WNDCLASS or WNDCLASSEX structure. Use the structure to register the class with a call to RegisterClass or RegisterClassEx, respectively. Use the structure to register the class with a call to RegisterClass or RegisterClassEx, respectively.

36. Create the Window Create a Window with a call to CreateWindow or CreateWindowEx. Create a Window with a call to CreateWindow or CreateWindowEx. Some of the parameters you will need to enter include: Some of the parameters you will need to enter include: –The name of a window class. –The name of the window. –A handle to a parent window. ( NULL if no parent) –Initial size and position of the window. CreateWindow and CreateWindowEx will return a window handle ( HWND ) if successful or NULL if failed. CreateWindow and CreateWindowEx will return a window handle ( HWND ) if successful or NULL if failed.

37. Showing the Window When we create a window it starts off hidden. When we create a window it starts off hidden. To show the window you must call ShowWindow (hWnd, nCmdShow) To show the window you must call ShowWindow (hWnd, nCmdShow) – hWnd – returned from CreateWindow / Ex. – nCmdShow – parameter from WinMain. –This will send a message to the window telling it to show. Call UpdateWindow (hWnd) to force the window to process the show message immediately. Call UpdateWindow (hWnd) to force the window to process the show message immediately.

38. Message Loops A message loop is needed to forward messages to their windows. A message loop is needed to forward messages to their windows. There are different ways to make a message loop but here is the most common one: There are different ways to make a message loop but here is the most common one: MSG msg = {0}; // returns FALSE when message is WM_QUIT while (GetMessage (&msg, NULL, 0, 0)) { TranslateMessage (&msg); // translate keyboard messages. DispatchMessage (&msg); // send message to its window. }

39. Message Loop (Cont’d) The loop uses: The loop uses: –A MSG structure – Holds message information. – GetMessage – Retrieves the next message. Waits for a message if the message queue is empty. – TranslateMessage – Translate some keyboard messages. – DispatchMessage – Dispatches the message to the appropriate windows procedure.

40. Windows Procedure Every window has an associate procedure for handling messages. Every window has an associate procedure for handling messages. Windows procedures have the following format: Windows procedures have the following format: – LRESULT CALLBACK ProcName (HWND hwnd, UINT msg, WPARAM wp, LPARAM lp);

41. Window Procedure Example Here is an example procedure: Here is an example procedure: LRESULT CALLBACK WndProc (HWND hwnd, UINT msg, WPARAM wp, LPARAM lp) { switch (msg) { case WM_DESTROY: PostQuitMessage (0); return 0; } return DefWindowProc (hwnd, msg, wp, lp); }

42. Window Procedure Parameters The procedure has 4 parameters: The procedure has 4 parameters: – HWND hwnd – handle to the window receiving the message. This is useful when multiple windows share the same procedure. You can think of it sort of like a this pointer. – UINT msg – The message. – WPARAM wp and LPARAM lp – These are parameters to the message and may represent different things.

43. Window Procedure Return When the procedure completely handles a message, it should return 0. When the procedure completely handles a message, it should return 0. The procedure should let the OS handle all other messages by calling DefWindowProc and then return its result. The procedure should let the OS handle all other messages by calling DefWindowProc and then return its result.

44. WM_DESTROY and WM_QUIT Initially we only want to handle one message: WM_DESTROY. Initially we only want to handle one message: WM_DESTROY. –This message is called when a window is closed. –Note that just because the window is destroyed, the program will still run. –To end the program we send WM_QUIT by calling PostQuitMessage.

45. WM_* Other messages you may want to handle include: Other messages you may want to handle include: – WM_CREATE – WM_SIZE – WM_LBUTTONDOWN – WM_LBUTTONUP – WM_MOUSEMOVE – WM_KEYDOWN – WM_KEYUP – WM_PAINT Check MSDN literature for information about these messages: http://msdn.microsoft.com/ Check MSDN literature for information about these messages: http://msdn.microsoft.com/http://msdn.microsoft.com/

46. Direct X The window has to obtain a handle to a Direct 3D and swap chain to render graphics objects in the windows object created. The window has to obtain a handle to a Direct 3D and swap chain to render graphics objects in the windows object created. In DirectX 11 this is based on the Programmable Pipeline. In DirectX 11 this is based on the Programmable Pipeline.

47. 23-Oct-1546 Suggested Reading http://msdn.microsoft.com/en-us/library/ee423531.aspx (Microsoft DirectX Graphics Documentation) http://msdn.microsoft.com/en-us/library/ee423531.aspx (Microsoft DirectX Graphics Documentation) http://developer.nvidia.com/page/cg_main.html (Cg homepage) http://developer.nvidia.com/page/cg_main.html (Cg homepage)

48. 23-Oct-1547 Programmable hardware pipeline on GPU Programmable hardware pipeline on GPU – concurrent processing on single chip Shaders offer code development flexibility and processing speed Shaders offer code development flexibility and processing speed Shader: program segment inserted into graphics pipeline and executed on GPU Shader: program segment inserted into graphics pipeline and executed on GPU – replaces section of fixed-function pipeline vertex shader, pixel shader, geometry shader vertex shader, pixel shader, geometry shader Variety of shader languages Variety of shader languages Summary