Embed Size (px)
Citation preview
Python and MongoDB at AHL
6th Python for Quant Finance Meetup
9 March 2015
Gary Collier
Opinions expressed are those of the author and may not be shared by all personnel of Man Group plc (‘Man’). These opinions are subject to change without notice, and are for information purposes only and do not constitute an offer or invitation to make an investment in any financial instrument or in any product to which any member of Man’s group of companies provides investment advisory or any other services. Any forward-looking statements speak only as of the date on which they are made and are subject to risks and uncertainties that may cause actual results to differ materially from those contained in the statements. Unless stated otherwise this information is communicated by Man Investments Limited and AHL Partners LLP which are both authorised and regulated in the UK by the Financial Conduct Authority.
© Man 2015 2
Legal Stuff
© Man 2015 3
Agenda
Python and MongoDB at AHL 1. AHL 2. Why Python At All? 3. Market Data Platform 4. Questions
10-15 mins
© Man 2015 4
AHL
The Platform Divide
© Man 2015 5
Why Python At All?
© Man 2015 6
Research vs Production
Market Data
Pla+orm
Opera0onal Tools Monitoring and BI
Quant Trading Strategies
Trade Execu0on System
Posi0on Management System
Production Trading Environment
Research Environment Market Data
Pla+orm
Visualisa0on Tools
IDE
Por+olio Builder
Simula0on Backtester
Model Toolbox
© Man 2015 7
The Platform Divide
Going Pure Python
© Man 2015 8
Get rid of the Divide
Market Data Storage
© Man 2015 9
© Man 2015 10
Time-Series Data
© Man 2015 11
Meta/Reference Data
MongoDB
© Man 2015 12
Bringing Together Disparate Data Sources
Dat
a A
PI
© Man 2015 13
Tick Store Architecture
Reuters
RM
DS
Mes
sage
Bus
Bloomberg
Banks
Kafka Queue
Kafka Queue
Kafka Queue
16 shard cluster Master + 1 replica
Linux 12 cores
256 GB RAM 96TB Disk
Infiniband network LZ4 compressed data
MongoDB Cluster
© Man 2014 14
Performance: 200 Future Markets
Previous Solution MongoDB
Trading Signal
Derived Data Item
Derived Data Item
Derived Data Item
Derived Data Item
Derived Data Item
Raw Data Items Raw Data
Items Raw Data Items Raw Data
Items Raw Data Item
Multi-user, versioned, interactive graph-based computation
© Man 2014 15
Graph Based Models
Source Data (Managed RDBMS)
Raw Data Items Raw Data
Items Raw Data Items Raw Data
Items Raw Data Item
Derived Data Item
Derived Data Item
Derived Data Item
Derived Data Item
Derived Data Item
Trading Signal
shard 1 shard 2 shard 3 shard 4 shard 1 shard 2 shard 3 shard 4 shard 1 shard 2 shard 3 shard 4
MongoDB Cluster ~1TB Data
~10,000 Stocks ~20 Years
250 Data Items Each Item is 600 MB Single model ~150GB Many Quants and models
© Man 2014 16
Monitoring and Logs as Data
© Man 2014 17
Black Box Diagnostics - Visualisation
© Man 2014 18
Kibana
