© M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.15.1 Sound effects and music Java Sound API: javax.sound.sampled 8- or 16-bit samples from

© M. Winter

COSC 3P91 – Advanced Object-Oriented Programming

5.1

Sound effects and music

Java Sound API: javax.sound.sampled• 8- or 16-bit samples• from 8,000Hz to 48,000Hz• mono or stereo• sound file formats: AIFF, AU, WAV(Examples: 16-bit, mono, 44,100Hz, WAV)

© M. Winter


5.2

Loading and playing a sound

try {File file = new File(“…”);AudioInputStream stream =

AudioSystem.getAudioInputStream(file);AudioFormat format = stream.getFormat();

} catch(Exception ex) {// IOException or UnsupportedAudioFileException

}try {

DataLine.Info info = new DataLine.Info(SourceDataLine.class,format);

line = (SourceDataLine) AudioSystem.getLine(info);line.open(format,bufferSize);

} catch(Exception ex) {// LineUnavailableException

}

© M. Winter


5.3

try {

int numBytesRead = 0;

while (numBytesRead != -1) {

numBytesRead = stream.read(buffer, 0, buffer.length);

if (numBytesRead != -1) {

line.write(buffer, 0, numBytesRead);

}

}

} catch (Exception ex) {

// IOException

};

line.drain();

line.close();

© M. Winter


5.4

Sound filter

Examples:• echo filter• simulated 3D sound filter

SoundFilter class (abstract class):• filter(byte[] buffer, int offset, int length) – Filters

an array of samples.• getRemainingSize() – Gets the remaining size, in bytes, that

this filter can play after the sound is finished.• reset() – Resets the filter so that it can be used again on a

different sound.

© M. Winter


5.5

FilteredSoundStream

public class FilteredSoundStream extends FilterInputStream {

…

public int read(byte[] samples, int offset, int length) throws IOException {

int bytesRead = super.read(samples, offset, length); if (bytesRead > 0) { soundFilter.filter(samples, offset, bytesRead); return bytesRead; };

if (remainingSize == REMAINING_SIZE_UNKNOWN) { remainingSize = soundFilter.getRemainingSize();

remainingSize = remainingSize / 4 * 4; };

© M. Winter


5.6

if (remainingSize > 0) {

length = Math.min(length, remainingSize);

for (int i=offset; i<offset+length; i++) {

samples[i] = 0;

};

soundFilter.filter(samples, offset, length);

remainingSize-=length;

return length;

} else {

return -1;

};

}

}

© M. Winter


5.7

Echo filter

• delay: numbers of samples to delay (44,100Hz sound, 1 sec delay = 44,100 samples)

• decay: value from 0 to 1– 0 means no echo– 1 means the echo is the same volume as the original

sound

Delay

Original sound First echo Second echo

© M. Winter


5.8

Emulating 3D sound

Many different effects are used to create 3D sounds.• Make sound diminish with distance so the farther away a

sound source is, the quieter it is.• Pan sounds to the appropriate speaker.• Apply room effects so sound waves bounce off walls, creating

echoes and reverberation.• Apply the Doppler effect so a sound source movement affect

its pitch.

© M. Winter


5.9

SoundManager

The SoundManager class has the following features:• provides a common interface for loading and playing sounds

(including filters), • extends the ThreadPool class,• each thread in the thread pool has its own buffer and line

object (thread-local variables).

© M. Winter


5.10

Playing music

Java Sound API provides MIDI sound capabilities in javax.sound.midi• synthesizes MIDI music through the use of a soundbank,• Java SDK includes a minimal-quality soundbank,• higher-quality soundbanks from http://java.sun.com/products/java-media/sound/

soundbanks.html

To play MIDI music, you need two objects:• Sequence object containing the data• Sequencer sending the Sequence to the MIDI synthesizer

© M. Winter


5.11

Sequence sequence = MidiSystem.getSequence(new File(“…”));

Sequencer sequencer = MidiSystem.getSequencer();

sequencer.open();

sequencer.setSequence(sequence);

sequencer.start();

Adding or taking away an instrument (track):

sequencer.setTrackMute(trackNum, true);

© M. Winter


5.12

Network programming

Socket-based communication (java.net.*)• server sockets (class ServerSocket)

– each server socket listens at a specific port– the server must be running before its clients initiate

contact– after the sever socket is contacted by a client, a

connection can be established– constructor ServerSocket(int port)

• client sockets (class Socket)– on the client side, constructors

• Socket(String host, int port)• Socket(InetAddress address, int port)

– on the server side• returned by the accept method of the server socket

– have an input and output stream

© M. Winter


5.13

Example (Server)...try {

ServerSocket server = new ServerSocket(10997);while (running) {

Socket client = server.accept();InputStream in = client.getInputStream();OutputStream out = client.getOutputStream();...// handle the client...in.close();out.flush();out.close();client.close();

};server.close();

} catch (Exception e) {e.printStackTrace();

};...

© M. Winter


5.14

Example (Client)

...

try {

Socket socket = new Socket(host,10997);

InputStream in = socket.getInputStream();

OutputStream out = socket.getOutputStream();

...

// Send and receive data

...

in.close();

out.flush();

out.close();

socket.close();

} catch (Exception e) {

e.printStackTrace();

};

...

© M. Winter


5.15

Network programming (cont’d)

The server in the previous example handles one client at a time.• just one client accepted at a time,• operations on streams are blocking,

Solution• use multiple threads (one for each client)

© M. Winter


5.16

Example (Server)

...

try {

ServerSocket server = new ServerSocket(10997);

while (running) {

Socket client = server.accept();

new ClientHandler(client).start();

};

server.close();

} catch (Exception e) {

e.printStackTrace();

};

...

© M. Winter


5.17

public class ClientHandler extends Thread {private Socket client;

public ClientHandler(Socket client) {this.client = client;

}

public void run() {try {InputStream in = client.getInputStream();OutputStream out = client.getOutputStream();...// handle the client...in.close();out.flush();out.close();client.close();

} catch (Exception e) {e.printStackTrace();};

}}

© M. Winter


5.18

Examplepublic class ChatterServer {

public static Vector clients = new Vector();

public static void main(String[] args) throws Exception { try {

ServerSocket server = new ServerSocket(10997);while (true) { Socket client = server.accept(); ClientHandler clientHandler = new ClientHandler(client); clients.add(clientHandler); clientHandler.start();};

} catch (Exception e) {e.printStackTrace();

};}

}

© M. Winter


5.19

Example

public class ClientHandler extends Thread {

private Socket client;private String clientName;private BufferedReader in;private PrintWriter out;

public ClientHandler(Socket client) { this.client = client; try {

in = new BufferedReader(new InputStreamReader( client.getInputStream()));

out = new PrintWriter(new OutputStreamWriter(

client.getOutputStream())); } catch (Exception e) {

e.printStackTrace(); };}

© M. Winter


5.20

Example

public String getClientName() {return clientName;

}

public synchronized void sendMessage(String msg) {if (out != null) {

out.println(msg);out.flush();

};}

public void broadcastMessage(String msg) { for(Iterator it = ChatterServer.clients.iterator(); it.hasNext();) {

ClientHandler ch = (ClientHandler) it.next();if (ch != this) ch.sendMessage(msg);

};}

© M. Winter


5.21

Example

public void run() { if (in != null && out != null) {

... while (true) {

String str = in.readLine();if (str == null) break;if (str.trim().equals("BYE")) break;broadcastMessage("From " + clientName + ": " + str);

}; broadcastMessage("User " + clientName + " has closed

connection."); client.close(); ChatterServer.clients.remove(this);...

}

© M. Winter


5.22

Class InetAddress

This class represents internet addresses:

• InetAddress addr = InetAddress.getByName("sandcastle"));

• byte[] b = {(byte) 139, (byte)57, (byte)96, (byte)6};

InetAddress addr = InetAddress.getByAddress(b)

© M. Winter


5.23

JDK 1.4 NIO Libraries

Channels (java.nio.channels.*)• Interfaces

– Channel

public void close();

public boolean isOpen();– ReadableByteChannel and WritableByteChannel

public int read(ByteBuffer dst);

public int write(ByteBuffer src);– ByteChannel– GatheringByteChannel and ScatteringByteChannel

long write(ByteBuffer[] srcs);

long write(ByteBuffer[] srcs, long offset, long length);

long read(ByteBuffer[] dsts);

long read(ByteBuffer[] dsts, long offset, long length);

© M. Winter


5.24

NIO (cont.)

• Classes– ServerSocketChannel

ServerSocketChannel sSockChan;

sSockChan = ServerSocketChannel.open();

sSockChan.configureBlocking(false);

InetAddress addr = InetAddress.getLocalHost();

sSockChan.socket().bind(new InetSocketAddress(addr,PORT));

– SocketChannel• SocketChannel has all the methods of ServerSocketChannel• methods for reading and writing• methods for managing the connection

© M. Winter


5.25

NIO (cont.)

sSockChan.accept(); (server)

SocketChannel.open(SocketAddress address); (client)or

channel = SocketChannel.open();

channel.connect(SocketAddress address);

– DatagramChannel• using UDP (User Datagram Protocol) instead of TCP• UDP is an unreliable protocol.

© M. Winter


5.26

NIO (cont.)Buffers• Classes

– ByteBuffer– CharBuffer– DoubleBuffer– FloatBuffer– IntBuffer– LongBuffer– ShortBuffer– MappedByteBuffer– methods for querying and manipulating the capacity,

position, limit, and mark of the buffer

0 1 2 3 4 5 6 7 8 9

mark

position

limit

capacity

© M. Winter


5.27

NIO (cont.)FileChannel.read(ByteBuffer src, int o, int length);

SocketChannel.write(ByteBuffer src);• clear() – preparing a buffer for filling

• flip() – preparing the buffer for draining

0 1 2 3 4 5 6 7 8 9

position

limit

capacity

0 1 2 3 4 5 6 7 8 9

position

limit

capacity

0 1 2 3 4 5 6 7 8 9

position

limit

capacity

© M. Winter


5.28

NIO (cont.)• rewind() – preparing the buffer for another draining

• compact() – moves the elements between the current position and the limit to the beginning of the buffer

• direct versus nondirect buffers

0 1 2 3 4 5 6 7 8 9

position

limit

capacity

0 1 2 3 4 5 6 7 8 9

position

limit

capacity

© M. Winter


5.29

Selector and SelectionKey

Provide a mechanism for multiplexing access to channels.• register a channel (along with a set of operations that the

selector should watch for)chan.register(readSelector,

SelectionKey.OP_READ, new StringBuffer());

– OP_ACCEPT– OP_CONNECT– OP_READ– OP_WRITE

• watching the channelsreadSelector.selectNow();– select() blocks until at least one channel has

activity– select(long timeout) as select() with timeout– selectNow() returns immediately

© M. Winter


5.30

Selector (cont.)

• accessing the channelsreadSelector.selectedKeys();– returns a Set of SelectionKeys– use an Iterator to get the keys– key.channel() returns the channel

© M. Winter


5.31

Example: ChatterBox

ChatterServer is a multi-user chat application that allows for any

number of users to connect to the server and send text messages to one

another.

The server needs to perform the following main functions:• Accept client connections• Read messages from clients• Write messages to clients• Handle disconnections (graceful or otherwise)

© M. Winter


5.32

ChatterServer

• Important attributes:

private ServerSocketChannel sSockChan;

private Selector readSelector;

private LinkedList clients;

private ByteBuffer readBuffer;

private ByteBuffer writeBuffer;

private CharsetDecoder asciiDecoder;

private Logger log = Logger.getLogger(ChatterServer.class);

© M. Winter


5.33

private static final int PORT = 10997; private void initServerSocket() {

try { // open a non-blocking server socket channel sSockChan = ServerSocketChannel.open(); sSockChan.configureBlocking(false); // bind to localhost on designated port InetAddress addr = InetAddress.getLocalHost(); sSockChan.socket().bind(

new InetSocketAddress(addr, PORT)); // get a selector for multiplexing the client channels readSelector = Selector.open();}catch (Exception e) { log.error("error initializing server", e);}

}

© M. Winter


5.34

private void acceptNewConnections() {try { SocketChannel clientChannel; while ((clientChannel = sSockChan.accept()) != null) { addNewClient(clientChannel);

log.info("got connection from: " + clientChannel.socket().getInetAddress());

sendBroadcastMessage("login from: " + clientChannel.socket().getInetAddress(), clientChannel);

sendMessage(clientChannel, "\n\nWelcome to ChatterBox, there are " + clients.size() + " users online.\n"); sendMessage(clientChannel, "Type 'quit' to exit.\n");

}}catch (IOException ioe) { log.warn("error during accept(): ", ioe);}catch (Exception e) { log.error("exception in acceptNewConnections()", e);}

}

© M. Winter


5.35

private void addNewClient(SocketChannel chan) {

// add to our list

clients.add(chan);

// register the channel with the selector

// store a new StringBuffer as the Key's attachment for

// holding partially read messages

try {

chan.configureBlocking( false);

SelectionKey readKey = chan.register(readSelector, SelectionKey.OP_READ, new

StringBuffer());

}

catch (ClosedChannelException cce) {

}

catch (IOException ioe) {

}

}

© M. Winter


5.36

private void sendMessage(SocketChannel channel, String mesg) {prepWriteBuffer(mesg);channelWrite(channel, writeBuffer);

} private void sendBroadcastMessage(String mesg, SocketChannel from) {

prepWriteBuffer(mesg);Iterator i = clients.iterator();while (i.hasNext()) { SocketChannel channel = (SocketChannel)i.next(); if (channel != from) channelWrite(channel, writeBuffer);}

}

private void prepWriteBuffer(String mesg) {writeBuffer.clear();writeBuffer.put(mesg.getBytes());writeBuffer.putChar('\n');writeBuffer.flip();

}

© M. Winter


5.37

private void channelWrite(SocketChannel channel, ByteBuffer writeBuffer) {long nbytes = 0;long toWrite = writeBuffer.remaining();// loop on the channel.write() call since it will not necessarily// write all bytes in one shottry { while (nbytes != toWrite) {

nbytes += channel.write(writeBuffer);try { Thread.sleep(CHANNEL_WRITE_SLEEP);}catch (InterruptedException e) {}

}}catch (ClosedChannelException cce) {}catch (Exception e) {} // get ready for another write if neededwriteBuffer.rewind();

}

© M. Winter


5.38

private void readIncomingMessages() {

try {

readSelector.selectNow();

Set readyKeys = readSelector.selectedKeys();

Iterator i = readyKeys.iterator();

while (i.hasNext()) {

SelectionKey key = (SelectionKey) i.next();

i.remove();

SocketChannel channel = (SocketChannel) key.channel();

readBuffer.clear();

long nbytes = channel.read(readBuffer);

if (nbytes == -1) {

log.info("disconnect: " + channel.socket().getInetAddress() + ",

end-of-stream");

channel.close();

clients.remove(channel);

sendBroadcastMessage("logout: " +

channel.socket().getInetAddress() , channel);

}

© M. Winter


5.39

else {

StringBuffer sb = (StringBuffer)key.attachment();

readBuffer.flip( );

String str =

asciiDecoder.decode(readBuffer).toString();

readBuffer.clear( );

sb.append( str);

String line = sb.toString();

…

sendBroadcastMessage(channel.socket().getInetAddress() +

": " + line, channel);

…

}

}

© M. Winter


5.40

ChatterClient

The ChatterClient needs to perform three tasks:• Connect to the server• Send messages• Receive messages

The client uses two threads, one for console input/writing and the main

execution thread.

© M. Winter


5.41

private void connect(String hostname) {try { readSelector = Selector.open(); InetAddress addr =

InetAddress.getByName(hostname); channel = SocketChannel.open(new

InetSocketAddress(addr, PORT)); channel.configureBlocking(false); channel.register(readSelector, SelectionKey.OP_READ,

new StringBuffer());}catch (UnknownHostException uhe) {}catch (ConnectException ce) {}catch (Exception e) {}

}

© M. Winter


5.42

Multi-Player Game Server Framework

Design goals:• Simplicity• Versatility/extensibility• Scalability• Performance

Goals are achieved by:• Encapsulate the communication messages into GameEvents.• Cleanly separate logic from mechanism.• Separate the server process into various stages of a pipeline.• Back each stage with a thread pool for concurrent operations.• Separate each stage with a queue to buffer events.• Provide a game logic plug-in architecture for supporting

multiple games within the same server application.

© M. Winter


5.43

Multi-Player Game Server Framework

Server Processes Event

Server Parses Event

Server Generates Event

Server Sends Event

Server Receives Event

Client Sends Event

Client Generates Event

Client Connect

Server Accept

Client Receives Event

© M. Winter


5.44

GameEvents

Selecting an event format:• Size/compactness• Human readability• Serialization/parsing ease and speed• Size of serialization/parsing code• Flexibility

Common options:• Serialized Java Objects• XML• Custom Binary• Binary-Encoded XML

© M. Winter


5.45

GameEvents

In addition to the GameEvent interface, an interface that will allow for the

passing of events between stages of the server’s event handling pipeline

is required.

public interface EventHandler {

public void handleEvent(GameEvent event);

}

Over-the-Wire event protocol:ClientId

(4 bytes)

GameName

(4 bytes)

PayloadSize

(4 bytes)

Payload

(PayloadSize bytes)

© M. Winter


5.46

GameControllers

The abstract class GameController encapsulates the server-side game

logic.• is able to have a variety of different games running

simultaneously in one GameServer. • initController(GameConfig gc): This method is for

concrete GameControllers to perform any initialization that they require.

• getGameName(): Must return the name of the game.• createPlayer(): Factory method that must return an object

that implements the Player interface.• createGameEvent(): Factory method that must return an

object that implements the GameEvent interface.

© M. Winter


5.47

GameServer and SelectAndRead

The GameServer class is the centerpiece of the server application and

contains the application’s main() method. The duties are• accept incoming socket connections,• keep track of connected clients,• manage the GameControllers.

The class SelectAndRead houses the selector that multiplexes client

channels and reads GameEvents from those channels.• complete GameEvents are passed off to GameControllers

based on the GameName hash code found in the event header.

© M. Winter


5.48

Server Framework Class Diagram

GameController

<<interface>> EventHandler

EventWriter

Wrap

EventQueue

GameServer SelectAndRead1

*

© M. Winter


5.49

A sample game (server)

GameController

<<interface>> EventHandler

EventWriter

Wrap

EventQueue

GameServer

SelectAndRead1

*

RPSController

GameEventDefault PlayerDefault

1

* *

<<interface>> GameEvent

<<interface>> Player

Documents

© M. Winter COSC 3P91 – Advanced Object-Oriented Programming 5.15.1 Sound effects and music Java Sound API: javax.sound.sampled 8- or 16-bit samples from