www.codingthearchitecture.com

Moudud Ahmed

Low Latency and High Volume Architecture

Contents Business Problem NFR Challenges Solution Technology overview Performance Monitoring Approach Outcome Lessons Learned

Business problem

OverviewCurrent Situation

Interest Rate SwapsManual ProcessesMultiple sourcesTrading Mechanism risk to dealers

OverviewCurrent Market Environment

Introducing change in a volatile environmentChange vs execution confidence

Credit Crunch hits high streetCredit Crunch hits high street

Volatile markets

Volatile markets…………

‘‘Market can findMarket can findsolutions to sub primesolutions to sub primedebt marketsdebt markets’’

LiquidityHub - Overview

NFR - Key ChallengesPerformance

Trade negotiation is time-sensitive, < 100 ms latency requiredYear one volume 20,000 msg/s scaling to 100,000 msg/s by year five.

ReliabilityNo loss of trade messages / transactions

Ease of IntegrationDealer & Distributors

Availability99.99%

Time to MarketGo Live in 9 Months

Solution

Solution – Functional Overview

Technology - Pricing Example

Data Tier

Database

Middleware TierMiddleware Server Middleware Server

Gateway Tier

Quote Gateway Market Data GatewayTrade Gateway

Application Tier

TradeServices

AdminServices

PricingServices

ReportingServices

Dealers(INVESTMENT BANKS)

Distributors(REUTERS & BBG)

11

1. Dealer streaming in prices1. Dealer streaming in prices

222. Payload transformed and sent to2. Payload transformed and sent topricing servicespricing services

Market Data

(Composite Price)

3. Generates composite prices,3. Generates composite prices,twice every secondstwice every seconds

33

Technology - Products

Data Tier

Database

Middleware TierMiddleware Server Middleware Server

Gateway Tier

Quote Gateway Market Data Gateway

Application Tier

TradeServices

AdminServices

PricingServices

ReportingServices

Trade GatewayCameron US Cameron MDS

Fiorano JMS Fiorano JMS

Spring Spring Spring Spring

Oracle RAC

BEA WLRT BEA WLRT

BEA WLRT BEA WLRT

BEA WLRT BEA WLRT BEA WLRT BEA WLRT

Technology - Hardware

Trade & QuoteGateway

Market DataGateway

Middleware

Trade Services Pricing Services Admin Services Reporting Services

Op/Trade Databases

Tick/ReportingDatabases

HP DL360Dual Core 2xCPU 3GHz4GB RAM

HP DL580Dual Core 4xCPU 3.4GHz8GB RAM

KeyCameron GatewayCameron Gateway

WLRTWLRT

Fioarano Fioarano JMSJMSWLRTWLRT

SpringSpringWLRTWLRT

OracleOracleRACRAC

Technology - Product PerformanceCameron Universal & Market data ServerFix Servers

FIX is a standard communications protocol within the InvestmentBanking domain but is flexible enough to be customised

Cameron US/MDS can handle > 25,000 msg/s

Fiorano MQPure JMS ServerCan handle > 80,000 non-persistent msg/s

Technology - Product PerformanceWebLogic Real Time provided a huge performanceimprovement compared to standard Sun JVM

With – 15ms latencyWithout - significantly > 100ms latency

Underpinned key components in each tierFix Gateway ServersJMS ServersSpring

Made Java solution a feasible and good choice forthis systemSystem handling ~ 25,000 msg/s with 95th percentile latency~15ms

Performance Monitoring

Data Tier

Database

Middleware TierMiddleware Server Middleware Server

Gateway Tier

Quote Gateway Market Data Gateway

Application Tier

TradeServices

AdminServices

PricingServices

ReportingServices

Trade Gateway

JMX Monitoring Framework

MBean MBean

MBean MBean MBean

MBean

11

1. 1. MBean MBean Server for eachServer for eachcomponent/servicecomponent/service

2. Admin Services retrieves all2. Admin Services retrieves allstatistics periodicallystatistics periodically

22 MonitoringWeb App

3. Monitoring Web App displays3. Monitoring Web App displaysstatistics to usersstatistics to users

33

Performance MonitoringAdministration application provides detailedstatistics:

LatencyPer message typePer Component if required

ThroughputService/Server Status

Provides an easy to understand dashboard tothe LiquidityHub Staff.

Approach Reference Architecture Design completed in horizontal and vertical 4 architects, 4 ??

Responsibilities

Hands on ?

Expertise in the correct area

Outcome• Designed, built and delivered the system in the9 months

System meets year 1 peak performance targets Current latency figures 4 – 8ms

System can scale to much higher messagevolumes

Lessons Learned Create more testing tool

Distributor GUI test harness

Log search and view

Early prototyping helped de-risk the architecture

Consider virtualisation

Managing 17 organisations is not easy Weekly tech forums worked well

Dedicated relationship manager

BreakBreak

Websitehttp://www.codingthearchitecture.com

London User GroupMonthly mix of presentation and discussion

Google Grouphttp://groups.google.com/codingthearchitecture