1. CSCE 552 Spring 2009 Network/Multiplayer and Audio By Jijun Tang

2. Announcements - 1 April 20 th (Monday): third presentation  Expected: working demo from start to finish, with at least one level finished  You can send me your current code so I can play it a bit Homework 4  Evaluate the best and worst games with AI, strength and weakness  6 points total  Due April 15th

3. Announcements - 2 Public Demo  2:00pm, May 5 th, Tuesday  Location: TBA Set up  Show the design (ppt)  Show the game (demo)  Allow others to play

4. PathPlannerApp Demo

5. Characteristics of Grids Fast look-up Easy access to neighboring cells Complete representation of the level

6. Waypoint Graph A waypoint graph specifies lines/routes that are “ safe ” for traversing Each line (or link) connects exactly two waypoints

7. Navigation Meshes (continued)

8. Random Trace (continued) How will Random Trace do on the following maps?

9. Overall Structure of the Algorithms 1. Create start point node – push onto open list 2. While open list is not empty A. Pop node from open list (call it currentNode) B. If currentNode corresponds to goal, break from step 2 C. Create new nodes (successors nodes) for cells around currentNode and push them onto open list D. Put currentNode onto closed list

10. Breadth-First Finds a path from the start to the goal by examining the search space ply-by-ply

11. Best-First (continued)

12. A* Uses both heuristic cost and given cost to order the open list Final Cost = Given Cost + (Heuristic Cost * Heuristic Weight)

13. Start Node and Costs F=G+H

14. Cost Map

15. Multiplayer Modes: Event Timing Turn-Based  Easy to implement  Any connection type Real-Time  Difficult to implement  Latency sensitive

16. Multiplayer Modes: Shared I/O Input Devices  Shared keyboard layout  Multiple device mapping Display  Full Screen and Screen Swap  Split Screen

17. Split Screen

18. Multiplayer Modes: Connectivity Non Real-Time  Floppy disk net  Email  Database Direct Link  Serial, USB, IrD, … (no hops) Circuit Switched (phones)  Dedicated line with consistent latency Packet Switched  Internet  Shared pipe

19. Protocols: Protocol Design Packet Length Conveyance Acknowledgement Methodology Error Checking / Correcting Compression Encryption Packet Control

20. Protocols: Packets Packets  Header = Protocol Manifest  Payload Gottchas  Pointers  Large/Variable Size Arrays  ADTs  Integer Alignment  Endian Order  Processor dependant Intrinsic Types (int and long)  Unicode vs. ASCII Strings

21. Protocols: Request for Comments RFC web site  http://www.rfc-editor.org/ http://www.rfc-editor.org/ Protocol Specifications  Definitive Resource  Public Criticism  Future Protocols

22. Example Packets IP Packet TCP Packet

23. Protocol Stack: Open System Interconnect

24. Another View

25. Protocol Stack: Physical Layer Bandwidth  Width of data pipe  Measured in bps = bits per second Latency  Travel time from point A to B  Measured in Milliseconds  You cannot beat the physics The Medium  Fiber, FireWire, IrD, CDMA & other cell Serial USB 1&2 ISDNDSLCable LAN 10/100/1G BaseT Wireless 802.11 a/b/g Power Line T1 Speed (bps) 20K 12M 480M 128k 1.5M down 896K up 3M down 256K up 10M 100M 1G b=11M a,g=54M 14M1.5M Table: Max Bandwidth Specifications

26. Protocol Stack: Data Link Layer Serializes data to/from physical layer Network Interface Card (NIC)  Ethernet  MAC Address

27. Protocol Stack: Network Layer Packet Routing  Hops  Routers, Hubs, Switches Internet Protocol (IP)  Contains Source & Destination IP Address  IPv4 Widespread Infrastructure  IPv6 Larger IP address

28. Protocol Stack: Network Layer: IP Address Unicast  Static  DHCP Multicast  Requires multicast capable router Broadcast  Local  Directed Loop Back  Send to self

29. Multicast and Broadcast

30. Protocol Stack: Network Layer: DNS Domain Name Service  Converts text name to IP address  Must contact one or more DNS servers to resolve  Local cache resolution possible Game Tips  Store local game cache to use when DNS out of order.  DNS resolution often slow, use cache for same day resolution.

31. DNS

32. Protocol Stack: Transport Layer Manage data deliver between endpoints  Error recovery  Data flow TCP and UDP used with IP  Contains Source and Destination Port  TCP is connection-oriented, UDP is connectionless Port + IP = Net Address  Port Range = 0-64k  Well known Ports 0-1k

33. Protocol Stack: Transport Layer: TCP Guaranteed Correct In Order Delivery  Acknowledgement system: Ack, Nack, Resend  Checksum  Out of Band Connection Required  Packet Window  Packet Coalescence (united)  Keep Alive Streamed Data  User must serialize data

34. Protocol Stack: Transport Layer: UDP Non Guaranteed Delivery  No Acknowledgement system  May arrive out of order  Checksum Not Connected  Source not verified  Hop Count Limit = TTL (time to live)  Required for Broadcasting Datagram  Sent in packets exactly as user sends them

35. TCP/UDP Comparison

36. Protocol Stack: Session Layer Manages Connections between Apps  Connect  Terminate  Data Exchange Socket API live at this layer  Cross platform  Cross language

37. Protocol Stack: Session Layer: Sockets Based on File I/O  File Descriptors  Open/Close  Read/Write Winsock  Provides standard specification implementation plus more  Extension to spec prefixed with “ WSA ”  Requires call to WSAStartup() before use  Cleanup with WSAShutdown()

38. Protocol Stack: Session Layer: Socket Design Modes  Blocking  Non-Blocking Standard Models  Standard  Select Extended Models  Windows WSAEventSelect I/O Completion Ports  Unix Poll Kernel Queues

39. Socket Programming

40. Socket Programming-2

41. Socket Programming -3

42. Socket Programming -4

43. Protocol Stack: Presentation Layer Prepares App Data for Transmission  Compression Pascal Strings String Tables Float to Fixed Matrix to Quaternion  Encryption  Endian Order When used cross platform or cross language  Serialize Pointers Variable Length Arrays

44. Endian Order

45. Protocol Stack: Presentation Layer: Buffering Packet Coalescence Induced Latency Dead Data Large Packets

46. Protocol Stack: Application Layer Handles Game Logic Update Models  Input Reflection  State Reflection Synchronization  Dead Reckoning  AI Assist  Arbitration

47. Real-Time Communications: Connection Models Broadcast  Good for player discovery on LANs Peer to Peer  Good for 2 player games Client / Server  Good for 2+ player games  Dedicated lobby server great for player discovery

48. Real-Time Communications: Peer to Peer vs. Client/Server BroadcastPeer/PeerClient/Server Connections0 Client = 1 Server = N N = Number of players BroadcastPeer/PeerClient/Server Send1N-1 Client = 1 Server = N ReceiveN-1 Client = 1 Server = N

49. Security: Encryption Goals Authentication Privacy Integrity

50. Security: Encryption Methods Keyed  Public Key  Private Key  Ciphers Message Digest Certificates IPSec (Internet Protocol Security)

51. IPSec

52. Security: Copy Protection Disk Copy Protection  Costly Mastering, delay copies to ensure first several months ’ sale  Invalid/Special Sector Read Code Sheets  Ask code from a line in a large manual Watermarking

53. Security: Execution Cryptography Code Obfuscation  Strip Symbols: int numOfPlayer;  int a1; Heap Hopper  Move sensitive data around in the heap to avoid snapshots Stack Overrun Execution  Check for buffer overflow Timer Hacking  Do not allow time to change backward DLL Shims

54. Privacy Critical data should be kept secret and strong encrypted:  Real name  Password  Address/phone/email  Billing  Age (especially for minors) Using public key for transforming user name and password

55. Security: Firewalls Packet Filter  Allow a packet based on its headers Proxies  Check contents Circuit Gateways  Setup secure sessions

56. Security: Firewalls: Network Address Translation

57. Security: Firewalls: NAT Traversal Port Forwarding Port Triggering DMZ Determining WAN IP

58. Summary: Further Study Socket Programming Serial Communication Server Design Network Gear & Infrastructure VOIP Tools of the Trade Unit & Public Beta Testing Middleware Databases Web Development Asynchronous Programming

59. Audio Programming

60. Audio in games is more important than ever before

61. Programming Basic Audio Most gaming hardware has similar capabilities (on similar platforms)  Mostly programming interfaces differ Learning fundamental concepts of audio programming is important

62. API Choices DirectSound (part of DirectX API)  Only available on Windows platforms  XACT for XNA OpenAL  Newer API  Available on multiple platforms Proprietary APIs  Typically available on consoles 3 rd Party Licensable APIs  Can offer broad cross-platform solutions

63. Analog Sound Wave

64. Basic Audio Terminology and Physics Amplitude  Measurement of a sound wave ’ s pressure Frequency  Measurement of the interval between wave cycles, typically measured in Hertz Pitch  The perception of frequency Tuning  Musical distribution of frequencies over keys Decibel  Measures sound amplitude

65. Digital Representation of a Sound Wave

66. Sampling Most common technique Sampling involves measuring the amplitude of the analog wave file at discrete intervals  The frequency of sampling is known as sampling rate  Each sample is typically stored in a value ranging from 4 to 24 bits in size  The size of the sample value in bits is known as the ‘ bit depth ’  Music CDs have a sample rate and bit depth of 44.1 kHz (samples/sec) and 16 bits (sample size)

67. Quantization Error in Sampling

68. Bit Depth and Signal Noise Bit depth of sample data affects signal noise  Signal to noise ratio = number of available bits / 1  For example, 8-bit samples have a 256:1 SNR (~48 dB), and 16-bit samples have a 65,536:1 SNR (~96 dB) Decibel ratio is calculated using 10 x log 10 (ratio) or 8.685890 x log e (ratio)

69. Sampling Frequency and Frequency Reproduction Sampling frequency affects range and quality of high-frequency reproduction Nyquist Limit  Frequencies up to one-half the sampling rate can be reproduced  Audio quality degrades as frequency approaches this limit

70. Sampling Errors vs. Sampling Frequency

71. Modern Audio Hardware Samples are piped into sound “ channels ”  Often a hardware pipeline from this point Various operations, such as volume, pan, and pitch may be applied 3D sounds may apply HRTF algorithms and/or mix the sound into final output buffers.

72. Sound Playback Techniques Two basic playback methods: 1. Play sample entirely from memory buffer 2. Stream data in real-time from storage medium Streaming is more memory efficient for very large audio files, such as music tracks, dialogue, etc Streaming systems use either a circular buffer with read-write pointers, or a double- buffering algorithm

73. Playback

74. Sample Playback and Manipulation Three basic operations you should know Panning is the attenuation of left and right channels of a mixed sound, results in spatial positioning within the aural stereo field Pitch allows the adjustment of a sample ’ s playback frequency in real-time Volume control typically attenuates the volume of a sound

75. Compressed Audio Format Compressed audio formats allow sound and music to be stored more compactly Bit reduction codecs generally are lightweight  ADPCM compression is implemented in hardware on all the major current video game console systems Psycho-acoustic codecs often have better compression  Discard sounds our ears would not typically be able to hear  Require substantially more computational horsepower to decode

76. MP3, Ogg Vorbis, Licensing & Patent Issues The MP3 format is patented  Any commercial game is subject to licensing terms as determined by Fraunhofer & Thompson Multimedia, the holders of the patents Ogg Vorbis is similar to MP3 in many ways  Open source and patent-free (royalty-free) Be aware of patent and license restrictions when using 3 rd party software

77. 3D Audio Two sets of data required when working in world coordinates:  Listener Data Composed of world position and orientation (virtual microphone in the world)  Source Data Composed of sound position, orientation, velocity, etc (virtual sound source in the world)

78. Listener/Source

79. Environmental Effects Environmental effects nearly always implemented in hardware Sound transmission is categorized in three ways  Direct transmission  Early reflections (echo)  Late reflections (reverberation)

80. Sound Transmission Categories

81. Environmental Effects Standards EAX 2.0 and beyond  EAX 2.0 developed by Creative Labs and released as an open standard  EAX 3.0 and 4.0 remain proprietary Creative Labs standards I3DL2  Open standard developed by IA-SIG, similar to EAX 2.0 in functionality

82. Programming Music Systems Two common music systems  MIDI-based systems (Musical Instrument Digital Interface)  Digital audio streaming systems (CD audio, MP3 playback, etc)

83. Advantages and Disadvantages of MIDI Actual music data size is negligible Easy to control, alter, and even generate in real-time High quality music is more difficult to compose and program Only effective if you can guarantee playback of a common instrument set

84. Example

85. Other MIDI-based technologies to be aware of DLS (DownLoadable Sound) Format  A standardized format for instrument definition files iXMF (Interactive eXtensible Music Format)  New proposed standard for a container format for interactive music

86. Advantages / Disadvantages of Digital Audio Streams Superb musical reproduction is guaranteed Allows composers to work with any compositional techniques Some potential interactivity is sacrificed for expediency and musical quality Generally high storage requirements

87. A Conceptual Interactive Music Playback System Divide music into small two to eight- bar chunks that we ’ ll call segments. A network of transitions from segment to segment (including loops and branches) is called a theme. Playing music is now as simple as choosing a theme to play. The transition map tracks the details.

88. Advanced Audio Programming 3D Audio Environmental Effects Integration Audio Scripting and Engine Integration Lip-sync Technology Advanced Voice Playback Voice Recognition

89. 3D Audio Environmental Effects Integration Environmental effects should be driven by a room ’ s shape and material composition.  Can determining the optimal effect settings be done automatically?  This may be important as game worlds become larger and more complex

90. 3D Audio Environmental Effects Integration (cont) Sound occlusion and damping is a particularly difficult problem to solve  This is essentially a pathfinding problem for audio.  Doors can dynamically affect a sound ’ s properties  Very few titles have even attempted a robust, general-purpose, and automated solution to these problems.

91. Room Acoustics

92. Dynamic Occlusion

93. Audio Scripting and Engine Integration Very little audio programming should be done by general game programmers Game Engine should offer robust support for audio triggers and scripts Engine should deal with audio scripts, not “ sound files ” Why is this so important?

94. Audio Scripting Many situations require much more information than can be embedded in a linear audio file  Sound Variation  Sound Repetition  Complex Sound Looping  Background Ambience

95. Lip-sync Technology Lip-sync technology is a blending of audio and visual techniques to create realistic- looking speech by in-game actors.  Simple techniques such as waveform amplitude measurement has worked previously, but …  In future titles, it will be considered inadequate.  Much work can still be done in this field.

96. Advanced Voice Playback Real-time spoken feedback is especially important in sports titles (simulated announcers) Game are reaching the limits of what current techniques (canned, prerecorded phrases combined in series) can provide. Again, this is an opportunity for future groundbreaking audio work.

97. Voice Recognition Spoken commands are much easier to deliver in certain situations. A great example of this? Squad-based tactical shooters. Current generation systems are still very error prone. A great opportunity for breakout audio technology.