1. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.15.1 Sound effects and music Java Sound API: javax.sound.sampled 8- or 16-bit samples from 8,000Hz to 48,000Hz mono or stereo sound file formats: AIFF, AU, WAV (Examples: 16-bit, mono, 44,100Hz, WAV)

2. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.25.2 Loading and playing a sound try { File file = new File(“…”); AudioInputStream stream = AudioSystem.getAudioInputStream(file); AudioFormat format = stream.getFormat(); } catch(Exception ex) { // IOException or UnsupportedAudioFileException } try { DataLine.Info info = new DataLine.Info(SourceDataLine.class,format); line = (SourceDataLine) AudioSystem.getLine(info); line.open(format,bufferSize); } catch(Exception ex) { // LineUnavailableException }

3. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.35.3 try { int numBytesRead = 0; while (numBytesRead != -1) { numBytesRead = stream.read(buffer, 0, buffer.length); if (numBytesRead != -1) { line.write(buffer, 0, numBytesRead); } } catch (Exception ex) { // IOException }; line.drain(); line.close();

4. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.45.4 Sound filter Examples: echo filter simulated 3D sound filter SoundFilter class (abstract class): filter(byte[] buffer, int offset, int length) – Filters an array of samples. getRemainingSize() – Gets the remaining size, in bytes, that this filter can play after the sound is finished. reset() – Resets the filter so that it can be used again on a different sound.

5. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.55.5 FilteredSoundStream public class FilteredSoundStream extends FilterInputStream { … public int read(byte[] samples, int offset, int length) throws IOException { int bytesRead = super.read(samples, offset, length); if (bytesRead > 0) { soundFilter.filter(samples, offset, bytesRead); return bytesRead; }; if (remainingSize == REMAINING_SIZE_UNKNOWN) { remainingSize = soundFilter.getRemainingSize(); remainingSize = remainingSize / 4 * 4; };

6. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.65.6 if (remainingSize > 0) { length = Math.min(length, remainingSize); for (int i=offset; i<offset+length; i++) { samples[i] = 0; }; soundFilter.filter(samples, offset, length); remainingSize-=length; return length; } else { return -1; }; }

7. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.75.7 Echo filter delay : numbers of samples to delay (44,100Hz sound, 1 sec delay = 44,100 samples) decay : value from 0 to 1 –0 means no echo –1 means the echo is the same volume as the original sound Delay Original sound First echo Second echo

8. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.85.8 Emulating 3D sound Many different effects are used to create 3D sounds. Make sound diminish with distance so the farther away a sound source is, the quieter it is. Pan sounds to the appropriate speaker. Apply room effects so sound waves bounce off walls, creating echoes and reverberation. Apply the Doppler effect so a sound source movement affect its pitch.

9. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.95.9 SoundManager The SoundManager class has the following features: provides a common interface for loading and playing sounds (including filters), extends the ThreadPool class, each thread in the thread pool has its own buffer and line object (thread-local variables).

10. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 10 Playing music Java Sound API provides MIDI sound capabilities in javax.sound.midi synthesizes MIDI music through the use of a soundbank, Java SDK includes a minimal-quality soundbank, higher-quality soundbanks from http://java.sun.com/products/java-media/sound/soundbanks.html To play MIDI music, you need two objects: Sequence object containing the data Sequencer sending the Sequence to the MIDI synthesizer

11. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 11 Sequence sequence = MidiSystem.getSequence(new File(“…”)); Sequencer sequencer = MidiSystem.getSequencer(); sequencer.open(); sequencer.setSequence(sequence); sequencer.start(); Adding or taking away an instrument (track): sequencer.setTrackMute(trackNum, true);

12. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 12 Network programming Socket-based communication ( java.net.*) server sockets (class ServerSocket ) –each server socket listens at a specific port –the server must be running before its clients initiate contact –after the sever socket is contacted by a client, a connection can be established –constructor ServerSocket(int port) client sockets (class Socket ) –on the client side, constructors Socket(String host, int port) Socket(InetAddress address, int port) –on the server side returned by the accept method of the server socket –have an input and output stream

13. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 13 Example (Server)... try { ServerSocket server = new ServerSocket(10997); while (running) { Socket client = server.accept(); InputStream in = client.getInputStream(); OutputStream out = client.getOutputStream();... // handle the client... in.close(); out.flush(); out.close(); client.close(); }; server.close(); } catch (Exception e) { e.printStackTrace(); };...

14. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 14 Example (Client)... try { Socket socket = new Socket(host,10997); InputStream in = socket.getInputStream(); OutputStream out = socket.getOutputStream();... // Send and receive data... in.close(); out.flush(); out.close(); socket.close(); } catch (Exception e) { e.printStackTrace(); };...

15. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 15 Network programming (cont’d) The server in the previous example handles one client at a time. just one client accepted at a time, operations on streams are blocking, Solution use multiple threads (one for each client)

16. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 16 Example (Server)... try { ServerSocket server = new ServerSocket(10997); while (running) { Socket client = server.accept(); new ClientHandler(client).start(); }; server.close(); } catch (Exception e) { e.printStackTrace(); };...

17. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 17 public class ClientHandler extends Thread { private Socket client; public ClientHandler(Socket client) { this.client = client; } public void run() { try { InputStream in = client.getInputStream(); OutputStream out = client.getOutputStream();... // handle the client... in.close(); out.flush(); out.close(); client.close(); } catch (Exception e) { e.printStackTrace(); }; }

18. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 18 Example public class ChatterServer { public static Vector clients = new Vector(); public static void main(String[] args) throws Exception { try { ServerSocket server = new ServerSocket(10997); while (true) { Socket client = server.accept(); ClientHandler clientHandler = new ClientHandler(client); clients.add(clientHandler); clientHandler.start(); }; } catch (Exception e) { e.printStackTrace(); }; }

19. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 19 Example public class ClientHandler extends Thread { private Socket client; private String clientName; private BufferedReader in; private PrintWriter out; public ClientHandler(Socket client) { this.client = client; try { in = new BufferedReader(new InputStreamReader( client.getInputStream())); out = new PrintWriter(new OutputStreamWriter( client.getOutputStream())); } catch (Exception e) { e.printStackTrace(); }; }

20. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 20 Example public String getClientName() { return clientName; } public synchronized void sendMessage(String msg) { if (out != null) { out.println(msg); out.flush(); }; } public void broadcastMessage(String msg) { for(Iterator it = ChatterServer.clients.iterator(); it.hasNext();) { ClientHandler ch = (ClientHandler) it.next(); if (ch != this) ch.sendMessage(msg); }; }

21. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 21 Example public void run() { if (in != null && out != null) {... while (true) { String str = in.readLine(); if (str == null) break; if (str.trim().equals("BYE")) break; broadcastMessage("From " + clientName + ": " + str); }; broadcastMessage("User " + clientName + " has closed connection."); client.close(); ChatterServer.clients.remove(this);... }

22. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 22 Class InetAddress This class represents internet addresses: InetAddress addr = InetAddress.getByName("sandcastle")); byte[] b = {(byte) 139, (byte)57, (byte)96, (byte)6}; InetAddress addr = InetAddress.getByAddress(b)

23. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 23 JDK 1.4 NIO Libraries Channels ( java.nio.channels.*) Interfaces –Channel public void close(); public boolean isOpen(); –ReadableByteChannel and WritableByteChannel public int read(ByteBuffer dst); public int write(ByteBuffer src); –ByteChannel –GatheringByteChannel and ScatteringByteChannel long write(ByteBuffer[] srcs); long write(ByteBuffer[] srcs, long offset, long length); long read(ByteBuffer[] dsts); long read(ByteBuffer[] dsts, long offset, long length);

24. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 24 NIO (cont.) Classes –ServerSocketChannel ServerSocketChannel sSockChan; sSockChan = ServerSocketChannel.open(); sSockChan.configureBlocking(false); InetAddress addr = InetAddress.getLocalHost(); sSockChan.socket().bind(new InetSocketAddress(addr,PORT)); –SocketChannel SocketChannel has all the methods of ServerSocketChannel methods for reading and writing methods for managing the connection

25. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 25 NIO (cont.) sSockChan.accept(); (server) SocketChannel.open(SocketAddress address); (client) or channel = SocketChannel.open(); channel.connect(SocketAddress address); –DatagramChannel using UDP (User Datagram Protocol) instead of TCP UDP is an unreliable protocol.

26. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 26 NIO (cont.) Buffers Classes –ByteBuffer –CharBuffer –DoubleBuffer –FloatBuffer –IntBuffer –LongBuffer –ShortBuffer –MappedByteBuffer –methods for querying and manipulating the capacity, position, limit, and mark of the buffer 0123456789 mark position limit capacity

27. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 27 NIO (cont.) FileChannel.read(ByteBuffer src, int o, int length); SocketChannel.write(ByteBuffer src); clear() – preparing a buffer for filling flip() – preparing the buffer for draining 0123456789 position limit capacity 0123456789 position limit capacity 0123456789 position limit capacity

28. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 28 NIO (cont.) rewind() – preparing the buffer for another draining compact() – moves the elements between the current position and the limit to the beginning of the buffer direct versus nondirect buffers 0123456789 position limit capacity 0123456789 position limit capacity

29. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 29 Selector and SelectionKey Provide a mechanism for multiplexing access to channels. register a channel (along with a set of operations that the selector should watch for) chan.register(readSelector, SelectionKey.OP_READ, new StringBuffer()); –OP_ACCEPT –OP_CONNECT –OP_READ –OP_WRITE watching the channels readSelector.selectNow(); –select() blocks until at least one channel has activity –select(long timeout) as select() with timeout –selectNow() returns immediately

30. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 30 Selector (cont.) accessing the channels readSelector.selectedKeys(); –returns a Set of SelectionKey s –use an Iterator to get the keys –key.channel() returns the channel

31. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 31 Example: ChatterBox ChatterServer is a multi-user chat application that allows for any number of users to connect to the server and send text messages to one another. The server needs to perform the following main functions: Accept client connections Read messages from clients Write messages to clients Handle disconnections (graceful or otherwise)

32. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 32 ChatterServer Important attributes: private ServerSocketChannel sSockChan; private Selector readSelector; private LinkedList clients; private ByteBuffer readBuffer; private ByteBuffer writeBuffer; private CharsetDecoder asciiDecoder; private Logger log = Logger.getLogger(ChatterServer.class);

33. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 33 private static final int PORT = 10997; private void initServerSocket() { try { // open a non-blocking server socket channel sSockChan = ServerSocketChannel.open(); sSockChan.configureBlocking(false); // bind to localhost on designated port InetAddress addr = InetAddress.getLocalHost(); sSockChan.socket().bind( new InetSocketAddress(addr, PORT)); // get a selector for multiplexing the client channels readSelector = Selector.open(); } catch (Exception e) { log.error("error initializing server", e); }

34. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 34 private void acceptNewConnections() { try { SocketChannel clientChannel; while ((clientChannel = sSockChan.accept()) != null) { addNewClient(clientChannel); log.info("got connection from: " + clientChannel.socket().getInetAddress()); sendBroadcastMessage("login from: " + clientChannel.socket().getInetAddress(), clientChannel); sendMessage(clientChannel, "\n\nWelcome to ChatterBox, there are " + clients.size() + " users online.\n"); sendMessage(clientChannel, "Type 'quit' to exit.\n"); } catch (IOException ioe) { log.warn("error during accept(): ", ioe); } catch (Exception e) { log.error("exception in acceptNewConnections()", e); }

35. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 35 private void addNewClient(SocketChannel chan) { // add to our list clients.add(chan); // register the channel with the selector // store a new StringBuffer as the Key's attachment for // holding partially read messages try { chan.configureBlocking( false); SelectionKey readKey = chan.register(readSelector, SelectionKey.OP_READ, new StringBuffer()); } catch (ClosedChannelException cce) { } catch (IOException ioe) { }

36. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 36 private void sendMessage(SocketChannel channel, String mesg) { prepWriteBuffer(mesg); channelWrite(channel, writeBuffer); } private void sendBroadcastMessage(String mesg, SocketChannel from) { prepWriteBuffer(mesg); Iterator i = clients.iterator(); while (i.hasNext()) { SocketChannel channel = (SocketChannel)i.next(); if (channel != from) channelWrite(channel, writeBuffer); } private void prepWriteBuffer(String mesg) { writeBuffer.clear(); writeBuffer.put(mesg.getBytes()); writeBuffer.putChar('\n'); writeBuffer.flip(); }

37. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 37 private void channelWrite(SocketChannel channel, ByteBuffer writeBuffer) { long nbytes = 0; long toWrite = writeBuffer.remaining(); // loop on the channel.write() call since it will not necessarily // write all bytes in one shot try { while (nbytes != toWrite) { nbytes += channel.write(writeBuffer); try { Thread.sleep(CHANNEL_WRITE_SLEEP); } catch (InterruptedException e) {} } catch (ClosedChannelException cce) { } catch (Exception e) { } // get ready for another write if needed writeBuffer.rewind(); }

38. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 38 private void readIncomingMessages() { try { readSelector.selectNow(); Set readyKeys = readSelector.selectedKeys(); Iterator i = readyKeys.iterator(); while (i.hasNext()) { SelectionKey key = (SelectionKey) i.next(); i.remove(); SocketChannel channel = (SocketChannel) key.channel(); readBuffer.clear(); long nbytes = channel.read(readBuffer); if (nbytes == -1) { log.info("disconnect: " + channel.socket().getInetAddress() + ", end-of-stream"); channel.close(); clients.remove(channel); sendBroadcastMessage("logout: " + channel.socket().getInetAddress(), channel); }

39. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 39 else { StringBuffer sb = (StringBuffer)key.attachment(); readBuffer.flip( ); String str = asciiDecoder.decode(readBuffer).toString(); readBuffer.clear( ); sb.append( str); String line = sb.toString(); … sendBroadcastMessage(channel.socket().getInetAddress() + ": " + line, channel); … }

40. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 40 ChatterClient The ChatterClient needs to perform three tasks: Connect to the server Send messages Receive messages The client uses two threads, one for console input/writing and the main execution thread.

41. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 41 private void connect(String hostname) { try { readSelector = Selector.open(); InetAddress addr = InetAddress.getByName(hostname); channel = SocketChannel.open(new InetSocketAddress(addr, PORT)); channel.configureBlocking(false); channel.register(readSelector, SelectionKey.OP_READ, new StringBuffer()); } catch (UnknownHostException uhe) { } catch (ConnectException ce) { } catch (Exception e) { }

42. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 42 Multi-Player Game Server Framework Design goals: Simplicity Versatility/extensibility Scalability Performance Goals are achieved by: Encapsulate the communication messages into GameEvent s. Cleanly separate logic from mechanism. Separate the server process into various stages of a pipeline. Back each stage with a thread pool for concurrent operations. Separate each stage with a queue to buffer events. Provide a game logic plug-in architecture for supporting multiple games within the same server application.

43. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 43 Multi-Player Game Server Framework Server Processes Event Server Parses Event Server Generates Event Server Sends Event Server Receives Event Client Sends Event Client Generates Event Client Connect Server Accept Client Receives Event

44. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 44 GameEvent s Selecting an event format: Size/compactness Human readability Serialization/parsing ease and speed Size of serialization/parsing code Flexibility Common options: Serialized Java Objects XML Custom Binary Binary-Encoded XML

45. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 45 GameEvent s In addition to the GameEvent interface, an interface that will allow for the passing of events between stages of the server’s event handling pipeline is required. public interface EventHandler { public void handleEvent(GameEvent event); } Over-the-Wire event protocol: ClientId (4 bytes) GameName (4 bytes) PayloadSize (4 bytes) Payload (PayloadSize bytes)

46. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 46 GameController s The abstract class GameController encapsulates the server-side game logic. is able to have a variety of different games running simultaneously in one GameServer. initController(GameConfig gc) : This method is for concrete GameController s to perform any initialization that they require. getGameName() : Must return the name of the game. createPlayer() : Factory method that must return an object that implements the Player interface. createGameEvent() : Factory method that must return an object that implements the GameEvent interface.

47. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 47 GameServer and SelectAndRead The GameServer class is the centerpiece of the server application and contains the application’s main() method. The duties are accept incoming socket connections, keep track of connected clients, manage the GameController s. The class SelectAndRead houses the selector that multiplexes client channels and reads GameEvent s from those channels. complete GameEvent s are passed off to GameController s based on the GameName hash code found in the event header.

48. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 48 Server Framework Class Diagram GameController > EventHandler EventWriter Wrap EventQueue GameServerSelectAndRead 1 *

49. © M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5. 49 A sample game (server) GameController > EventHandler EventWriter Wrap EventQueue GameServer SelectAndRead 1 * RPSController GameEventDefaultPlayerDefault 1 ** > GameEvent > Player