Distributed Trading System

SWE 622 Charles Beach

James WolfeScott Jones

Sharita Green

Agenda

Prototype Design Code Snippets Evaluations Logistic Measurements Demo

Evaluator

. . .

network

Exchange1

Trader1

System Design

ExchangeX

TraderX . . . Logs

TraderClientGoodsMap<good, price>

ExchangeEngine

network

Exec <main>

Buy/Sell

<RMI>Echo

Pub/Sub

Exchange

GUI

Trader

Exchange Design

Exec <main>

GUI

network

Exchange

ConfigFile

ExchangeInterface

Buy/Sell

<RMI>

Receive Echoes

<Pub/Sub>

Trader

TraderEngine

Trader Design

Communication

Traders communicate with Exchanges via RMI Register Buy Sell

Exchanges echo transactions to Traders via Pub/Sub.

Trader Registration

Configuration File TraderEngine

ExchangeInterface <Exchange1>

ExchangeInterface <Exchange2>

ExchangeEngine ExchangeEngine

network

TraderClient<Trader1>

TraderClient<Trader1>

Exchange1 Exchange2

RMI: register()

Trader1

Exchange Transaction Echoes

Exchange

network

ExchangeInterface

TraderEngine

TraderGUI

ExchangeInterface

TraderEngine

TraderGUI

Trader1 TraderN

RMI: buy/sellTraderX

Pub/Sub

Trader Registration

Date tempDate = new Date();long time1 = tempDate.getTime();

// RMI Register methodArrayList tempList = _bRemote.registerTrader(traderName);

tempDate = new Date();long time2 = tempDate.getTime();

// Calculate the clock offset long mid = (time2 + time1)/2; offset = ((Long)tempList.get(0)).longValue() - mid;

// Calculate the network latency long netLat = (time2 - time1)/2;

TraderLogger.instance().logExchangeTimeOffset(getExchangeName(), offset netLat);

Calculation of exchange/trader clock offset and network latency

Trader processing Echo

// Determine if transaction is the next one expectedif(sequenceNumber == client.getSequence()+1) { long transactionTime = timeStamp + client.getOffset(); client.incrementSquence();

// Determine if the data is stale if((_time - transactionTime) >= 10000) { updateGUI(exchangeName, function, product, quantity, price, true); } else { updateGUI(exchangeName, function, product, quantity, price, false); }

Detection out of order echo sequences and stale data

Exchange Processing of Registrations Update registration times as traders re-register Continuously check registration times for “failed” traders

public void registerTrader(String traderName) { ...

// timestamp registration Date d = new Date(); long registerTime = d.getTime();

// re-register by updating registration time client.setRegisterTime(registerTime); ExchangeLogger.instance().logTraderRegistration(traderName); ...}

class CheckTraders extends Thread { ... Iterator itr = _traders.iterator(); Date d = new Date(); long time = d.getTime(); while(itr.hasNext()) { TraderClient client = (TraderClient)itr.next(); if (time > client.getRegisterTime() + 10000) { ExchangeLogger.instance().logTraderFailure(client.getTraderName()); itr.remove(); } }

RMI register

Trader Exchange

(ET)

Tt1

Tt2

Trader CalculationsTte = (Tt1 + Tt2) / 2

Offset = Et - Tte

EtTte

Offset Calculation

Problem: Need to identify an un-received message as stale before subsequent “on time” message becomes stale itself.

Solution: Include in echo message sequence number, time of message and time of

previous message. Registration response provides Trader with the sequence number of last

message sent. Hold “out of order message” in queue until missing message(s) arrive or until

previous message(s) are stale. Previous message staleness calculated by taking “time of previous message”

from on hold message, adding offset and comparing against current time. When current time > 10 sec more than calculated time of previous message, then

all previous un-received messages are noted as stale and on hold message can be released (and guaranteed to not be stale).

Staleness and Out of Order Messages

Evaluations

Test Scenario included two Exchanges and two Traders across two machines connected by a local LAN

PC1

network

Exchange1

Trader1

PC2

Exchange2

Trader2

Evaluation 1

Latency between starting to send an order and proceeding with other activities.

Exchange logs a timestamp before echoing order and when finished notifying all clients

Exchange Number of Orders Processed

Average Latency

Exchange1 23 19ms

Exchange2 21 19ms

Evaluation 2

Latency between issuing an order and the last trader receiving it’s echo

Exchange logs a timestamp before echoing order Each trader logs a timestamp when receiving an order Evaluator computes traders timestamp relative to

exchanges based on offset of clocks.

Exchange Number of Orders Processed

Average Latency

Exchange1 23 91ms

Exchange2 21 99ms

Evaluation 3

Latency between trader failure and the last exchange stopping echoing orders.

Trader logs a timestamp when going offline Exchange logs a timestamp when detection of trader failure Evaluator computes traders timestamp relative to exchanges based on

offset of clocks.

Amount of time between trader failure and its next registration. At most 15 seconds

Evaluation 4

Storage footprint on exchange required to echo orders as a function of the number of traders

With our pub/sub approach, the exchange only sends out ONE message that reaches all Traders.

Therefore the storage footprint required to echo orders to traders was not expected to grow significantly with the number of Traders.

Function remains CONSTANT

Evaluation 4

Memory Footprint

The exchange used about:       31.3MB with no traders at the very start,           but then drops to 28.29MB if nothing happens      28.33MB when one trader was started      30.59MB on first sell request (still 1 trader)      30.72MB on second sell request (still 1 trader)      31.02MB on seven buy requests (still 1 trader) 31.75MB on resizing (bigger) the server window      32.14MB on resizing (smaller) the window      32.14MB when second trader started      32.16MB on first buy from second trader      32.21MB on first sell of new product from second trader      32.21MB when third trader started      32.28MB after a while with no input (it fluctuates at this point 32.27 - 32.33, but mainly 32.28)       32.38MB on first sell of new product from third trader

Conclusions Additional Traders have minimal impact on system memory

Evaluation 5

Average and max clock skew

Trader logs each calculated “time offset” relative to each exchange over 10 second intervals

Trader Exchange Max Skew Avg SkewTrader1 Exchange1 3ms 0ms

Exchange2 2ms 0ms

Trader2 Exchange1 2ms 0ms

Exchange2 4ms 0ms

Evaluation 6

Average and max network latency

Trader logs each calculated network latency during re-registration.

Trader Exchange Max Latency

Avg Latency

Trader1 Exchange1 8ms 4ms

Exchange2 7ms 5ms

Trader2 Exchange1 9ms 5ms

Exchange2 8ms 5ms

Development Measurements

SLOC count: 2009 Exchange: 473 Trader: 557 Logger: 175 Evaluator: 804

Design Effort: 110 man-hours Design: 30hrs Coding/Integrating: 75hrs Experiments/Evaluation: 5hrs

Demo

Questions