Nov 27, Fall 2006 IAT 410 1

Networks

Network ProtocolsPeer-to-peerClient-Server ConfigurationsTrust

Nov 27, Fall 2006 IAT 410 2

Networks Required for multiplayer games 3 Standard technologies

– Modems– Ethernet– Internet

Nov 27, Fall 2006 IAT 410 3

Internet The greatest thing since sliced bread The savior of humanity Will increase freedom and

democracy– Around the world– In your neighborhood

Nov 27, Fall 2006 IAT 410 4

Internet User Protocols TCP

– Connection– Reliable– Bytes arrive in order

they were sent– Collects small

packets and transmits them together

– Stream of bytes

UDP– Connectionless– Unreliable– Arbitrary arrival

order

Nov 27, Fall 2006 IAT 410 5

TCP Reliable stream of bytes

– Implies the need for a “connection” Connection sets up data structures

– Hold incoming packets– Hold outgoing packets– Handle retransmits

Nov 27, Fall 2006 IAT 410 6

TCP Reliability Each packet does Send-Receive-

Acknowledge

Sender holds sent packet until ACK is received

Sender retransmits if ACK takes too long

SenderReceive

ReceiverSend

Acknowledge

Nov 27, Fall 2006 IAT 410 7

TCP One Send-Receive-Ack takes time Overlay Sends and Acks Maintain a queue in sender and receiver

0

1

2

3

0

1

2

3

Ack

0

1

2

3

SendSender

Sender

Receiver

Nov 27, Fall 2006 IAT 410 8

TCP Circular Queue -- Sender Sends data and Puts it in send queue Sets timer on this queue item If timer expires, and no ACK, re-send

data– Set another, longer timer– Exponentially increasing time

When ACK received – If this queue slot is the oldest,

• Free the slot for new data If no queue space avail, sender app

waits!

Nov 27, Fall 2006 IAT 410 9

TCP Receive Queue Receiver maintains a queue the

same size as the sender’s When a packet arrives, send ACK If the packet is next in sequence

– Give it to application Else Keep it in queue

– Another, earlier packet is on its way

Nov 27, Fall 2006 IAT 410 10

TCP If no ACKS arrive for a long enough

time– Temporarily gives up– Sends test packets

When test packets get through– Starts slow, builds up

Nov 27, Fall 2006 IAT 410 11

TCP Wrap-up

Connection sets up sequencing and queues– Reliable arrival: Retransmit– Reliable order: Sequence numbers

TCP bunches up data on 200ms intervals– Minimizes overhead for small chunks of data– This option can be turned off

TCP Has an “emergency” channel– OOB Out Of Band

Nov 27, Fall 2006 IAT 410 12

UDP Connectionless!

– No underlying data to maintain Unreliable transmission

– If you lose a packet, it’s gone– Network software must handle this

Out-of-order arrival– Network software must handle that, too!

Fast– When the port gets the data, the app gets it

Nov 27, Fall 2006 IAT 410 13

UDP Packets will drop!

– 1 in 5 over non-local connection Have to do your own re-send Some packets are time sensitive

– Care little about the past ship location No header compression

– May end up with greater overhead than TCP with PPP

Nov 27, Fall 2006 IAT 410 14

Game Architectures Peer-to-peer Client/Server

– One server per game Floating server

– One client is also a server Distributed server

– Multiple servers for large world

Nov 27, Fall 2006 IAT 410 15

Peer-to-Peer Simple version: Lockstep

– eg. Doom– Each client transmits to other– Wait for everyone to get data– Proceed to next step

Nov 27, Fall 2006 IAT 410 16

Peer-to-Peer Advantages

– Simple– Nobody has to provide a

server• Including the Game’s

authors!– Good for turn-based games

with low bandwidth– TCP

Disadvantages– Frame rate is that of

• Slowest machine• Worst connection

– Hackable– Not good for real-time

games

Nov 27, Fall 2006 IAT 410 17

Client/Server Server per game

– MUDs, Fireteam, NetTrek– Someone must provide server ($$$)

• Possibly the game’s authors– Less hackable– Single point of failure– Server must be big & well-connected

Nov 27, Fall 2006 IAT 410 18

Floating Server Peer-to-peer Server resolves the action One peer is the server

– Unreal • One player elects to be the server

– X-Wing vs Tie-Fighter:• First player to enter session

– Starcraft• Player with the CD

Nov 27, Fall 2006 IAT 410 19

Multiple Server Many machines coordinate service

– Ultima Online, Everquest, AOL Used for large virtual worlds Everquest

– One server per game-geographic region– Freeze on handoff affects game play

Nov 27, Fall 2006 IAT 410 20

What Data to Send?

Sending entire world state is usually too much

Can send just user actions– Simulation engine does the same thing at

each client– Pseudo-random numbers from same seed

Nov 27, Fall 2006 IAT 410 21

Sending User Actions--Problems Any error in engine

– Divergence in worlds– Small error can lead to big divergence

X-Wing vs Tie Fighter– Created a resynchronize protocol– Causes jumps

• Wrote smoothing algorithm for resynchs Sim City 2000 Network Edition

– Send checksums for world state each turn

Nov 27, Fall 2006 IAT 410 22

Prediction Eg. Unreal Waiting for user inputs is too slow Client does prediction

– Motion prediction Server corrects things if client is

wrong

Nov 27, Fall 2006 IAT 410 23

Prediction: Dead Reckoning Eg. SIMNET (US Army Tank Simulator) Each vehicle simulates own tank Sends data every 5 seconds, updating

– Position, Speed, Acceleration– Expected path– Prediction violation criteria

Receiver simulates own tank– AND simulates local copy of other tanks

Nov 27, Fall 2006 IAT 410 24

Dead Recokoning Receiver gets latest 5-second update Updates own copy of other tanks Predicts other tanks

– Using prediction data– Until new data arrives

Each simulator also sends update– When own prediction violates own criteria

Assumes latencies < 500ms

Nov 27, Fall 2006 IAT 410 25

Dead ReckoningSim A

A’s Predicted Path

B’s Predicted Path

Sim B Sim A

A’s Predicted Path

B’s Predicted Path

Sim B

B Exceeds prediction: predict again and transmit

Transmit new prediction every 5 seconds

Predict B Predict BPredict A Predict A

Nov 27, Fall 2006 IAT 410 26

Dead Reckoning: Requirements Data structures for other entities Model of entity behavior

– Vehicle speed, acceleration range, turn radius

– Responsiveness to commands Situation parameters

– Following a road – Precomputed path (NPCs)

Nov 27, Fall 2006 IAT 410 27

Multiple Copies Maintain 2 Data sets Now

– Accurate self– Predicted others– “Zero” latency for self

Ground Truth– Accurate everybody– Large latency for almost everybody– 200-500ms ago

Nov 27, Fall 2006 IAT 410 28

Latency Issues When latencies get high

– Prediction gets worse and worse Correcting prediction errors may

cause visual jumps– Easy to notice!

If jumps are large enough– Temporarily interpolate between wrong

prediction and the new correction

Nov 27, Fall 2006 IAT 410 29

Prediction Interpolation

Real

InterpolatedResponse

Predicted

Nov 27, Fall 2006 IAT 410 30

Some games may allow distributed ownership– Ballistic simulation– Shooter fires bullet– Intended target receives the simulation

Sports - eg. Tennis– Player A hits ball– Player B gets simulation token – B simulates ball path from A’s racket

Token Ownership

Nov 27, Fall 2006 IAT 410 31

Trust “Never trust the client” Data on the user’s hard drive is

insecure– Diablo utility to modify character data– Wrote patch to prevent hacking

• Throws out your stuff if there’s a time inconsistency

– Daylight savings nuked my stuff!

Nov 27, Fall 2006 IAT 410 32

Trust Network communications are insecure NetTrek communications are encrypted NetTrek also requires “blessed” client

– Servers have different policies on requiring a blessed client

– Prevents robot players or assistants

Nov 27, Fall 2006 IAT 410 33

Trust -- Checksums First line of defense:

– Checksum of all packets– Include header in checksum!– Stops casual tampering

Hash function– Hard to compute source value from

result– MD5

Nov 27, Fall 2006 IAT 410 34

Checksums Not immune to:

– Code disassembly– Packet replay

Packet replay attack:– Capture a legal packet, and re-send it more

frequently than allowed– Client can restrict send frequency– Server cannot reject high-frequency packets

• Internet bunch-ups are source of OK bunch-ups

Nov 27, Fall 2006 IAT 410 35

Combating Replay Each new packet client sends is different

– Add a pseudo-random number to each packet– Not just sequence number!– Client & Server match pseudo-random numbers

Random numbers– Seeds must match!– Dropped packets: include sequence number!

Nov 27, Fall 2006 IAT 410 36

Combating Replay XOR each packet with a pseudo-

random bit pattern– Make sure the bit patterns are in sync!– Based on previous synchronized

pseudo-random numbers Add junk – Confuse length analysis

Nov 27, Fall 2006 IAT 410 37

Reverse Engineering Remove symbols Put encryption code in with rest of

network stuff Compute magic numbers:

– At runtime– In server

Encrypt from the start!

Nov 27, Fall 2006 IAT 410 38

Lists Of Servers Denial of service:

– Send a packet to server-server saying “I’m a server”

– Fake the IP return address with a random IP#

– Server-server adds “new server” to list– Server may run out of memory storing

hundreds of thousands of fake servers

Nov 27, Fall 2006 IAT 410 39

List of Servers Require a dialog

– Server-list server responds with • Password• Keepalive interval

– Password must be given by attacker at the correct time

– Works OK if client is not better connected!