From Back Testing to Live Trading

Vesna Straser, PhD

QuantCon, April 2016

Trade System Design Process

Trade Model and System Design

•Evaluate signal strength & expected alpha

•Check for model overfitting

•Check for look ahead biases

•Optimize system parameters

Back Testing

•Conduct out-of- sample testing

•Check for data consistency

•Evaluate system risk parameters

•Evaluate position liquidity

•Use simplified order fill and TC models

Simulation / Paper Trading

•Trade a strategy using tick data at simulated prices, trade cost, and fill rates

•Use market simulator that mirrors strategy type

Live Trading

•Commit capital to trade with real market data at actual prices, fill rates, and trade costs

Execution Testing Phase

Model Testing Phase

Performance Delta Between Back

Test, Simulation, and Live Trading

Two sets of factors:

Trade model / system specific → what and how much to trade

Bad data, over-fitted model, poor out of sample performance

Depends on signal research and modeling nuances

Trade execution specific → how to trade

Related to order fill assumptions: how orders are filled in terms of price, size,

and time

Can be responsible for large discrepancies in trading performance

High turnover strategies will tend to have greater discrepancies

Execution Factors Driving

Performance DeltaBack Testing Simulation Trading Live Trading

Data Structure OHLCV bar values

built from trade data

Bar or Tick Data

Market Depth

Level I Tick Data

Market Depth

Signal Calculation End of bar EOB or on every tick On every tick

Fill Price = f (order type) Fixed (OHLC) Simulated Market Driven

Fill Rate = f (order type) 100% Simulated ≤ 100% Market Driven ≤ 100%

Transaction Cost = f (order type) Fixed TC Model Market Driven

Trade Count = f (system settings) One trade per bar One or Market Driven Market Driven

Latency Not included Network Network and

Exchange

Strategy type and holding time

Example: One System, Two P&Ls

Mean Reversion

System using 10

minutes time

interval

Back Test stats:

o P&L: $188.5

o 6 trades

Simulation stats:

o P&L: $112.5

o 5 trades

Trade did not occur in Simulation because entry signal was not true in real time

Sim Trade exited at different time due to real time price dynamics

Sim Trade entry price is higher (worse) than Back Test Price

Comparing Price Slippage

Actual Trade. Price

Decision Price

Simulation Trade Price

Slippage

Signal = True

Decision

Time

Pric

e

TimeNext Bar

Open

BT Price Open

BT Price Close

Next Bar

Close

Bar 33 Bar 34 Bar 35 Bar 36 Bar 37

Comparing P&L Slippage

Next

OpenEntry

Time

TimeNext

Close

Next

Open

Next

Close

BT EnOPBT EnCP

AEnP

ST EnP

Entry = True

Pric

e

D EnP

BT ExOP

O P&L

ST ExP

S P&L

BT ExCP

C P&L

Exit

Time

Exit = True

D ExP

D P&L

Bar 33 Bar 34 Bar 35 Bar 36 Bar 37

AExP

A P&L

Timing Risk

Comparing Trade Count DynamicsP

ric

e

Time

S P&L

Bar 33 Bar 34 Bar 35 Bar 36 Bar 37

A P&L

O P&L

C P&L

Exit = True

Trade 1

BT Trade 1

Entry = True Exit = True

D P%L

Entry = True

Trade 2

Understanding Back Testing

Limitations

Signal is evaluated at the end of bar not intra-bar on every tick → lower/higher trade count

Actual price movement within a bar is not known → which order gets filled first?

Only one trade (entry or exit) per bar possible → lower trade count

MO are filled at OC prices (user specified) → slippage

LO are filled if limit price is within the HL range → 100% fill rate

In case of multiple active limit orders BT engine may fill all limit orders → overfills?

Inability to account for sudden price moves / gaps → higher fill rate or trade count

Simplified fill model: based on trade prices only (vs bid/ask data) → lower spread cost

100% fill rate if missing volume data → higher fill rate and lower market impact

Slippage is incorporated as fixed rate (in ticks / cents) → imprecise market impact

Missing network latency → higher fill rate and trade count

Improving Back Testing Accuracy

Incorporate intra-bar price movement assumptions:

If O is closer to H, assume O – H – L – C price move

If O is closer to L or in the middle assume O – L – H – C

Know which limit order gets filled first, “cancel” others

Hybrid OHLC and tick data approach: add tick data replay for bars where signal is true

Incorporate bid/ask data or spread to achieve more realistic fill price and spread cost

Buy market orders are filled at ask prices

Sell market orders are filled at bid prices

Trade price closest to the decision time (e.g., O) would generate more accurate slippage

If configurable, assume conservative fill model for limit orders (limit price vs HL)

Incorporate Volume data to fine tune fill rate and market impact assumptions:

Limit fill size to % of bar volume allowing for partial fills

Make market impact model a function of relative order size: order size / bar volume

Understanding Simulation Trading

Limitations Signals are processed using OHLC or tick data (replayed or in real time) → Is the signal processing timing in sync with BT?

Orders are processed using tick data but are filled per various order fill assumptions applied in market simulator

Not all market simulators use level 2 data in simulating order fills (Interactive Brokers is using level 1 only)

Order fill probability (price and size at which order is filled) is function of many dynamic variables:

Order type: market, limit, stop, pegged

Order size

Bid/Ask volume = visible + hidden

Market depth = visible + hidden

Trade volume

Time of day

Place in queue = f (number of limit orders in the queue)

Executing/routing exchange = f (liquidity, order processing latency, market conditions)

Latency = f (network, broker, exchange, market conditions) → may cause missed trades in live trading

Market conditions = f(volatility, trade rate, momentum, price gaps) → may cause missed trades in live trading

Impact of limit orders on market depth changes is difficult to capture

Improving Simulation Trading

Accuracy Use realistic limit order fill assumptions, e.g., limit order is filled when trade price:

touches a limit price → most liberal

trades through the limit price

trades through the limit price by X ticks → conservative

touches the limit price X times, where X = f (place in queue) → most realistic

fill price = Limit price → conservative

fill price = Trade Price → liberal, assumes price improvement

Market depth significantly improves accuracy of market order fill prices in case of partial fills

Use bid/ask volume for a more realistic market order fill rate

account for partial fills when order size > bid/ask size

Use customizable transaction cost model = f (order size, side, spread, volume, depth, volatility, horizon)

Use dynamic latency model = f (real time network latency, avg exchange latency, avgorder processing latency)

Defining Strategy Holding Time

milliseconds seconds minutes hours days weeks

HIGH FREQUENCY

HT = milliseconds to seconds Expected Profit < 1 spread Order size = very small

Cost sensitivity: very high Trading concern = speed Strategy types:

Latency arbitrage Statistical arbitrage Market making Pairs/spread trading

INTRA DAY

HT = minutes to hours Expected profit = x * spread Order size = small

Cost sensitivity: high Trading concern = spread

capture Strategy types:

TA pattern following Momentum/Reversal Scalping

News

LOW FREQUENCY

HT = days to months Expected profit > 1% Order size = larger

Cost sensitivity: moderate Trading concern = liquidity Strategy types:

Long / Short Fundamental Factors Earnings quality

Back Testing and Simulation Model

Selection

HFT

INTRA DAY

LOW FREQUENCY

Back Testing = Simulation Model

Use level 2 data

Need custom built simulator; off the shelf products do not capture sufficient details

Custom TC model = f (trade strategy)

Latency is critical

Use O B/A based prices to fill market orders

Apply intra-bar price move assumptions

Use bar volume data

Apply spread based TC model

Use level 2 data

Use conservative limit order fill model

Apply small order TC model

Account for latency

Level 1 is sufficient, level 2 is better

Apply order size limits

Use complex order types (e.g., VWAP)

Apply large (parent) order TC model

Use conservative prices to fill orders, e.g., (H+L)/2, VWAP (if orders are large)

Use bar volume data to limit order size or estimate execution horizon

Apply conservative TC assumptions

BACK TEST SIMULATION

Takeaways

1. Pick BT and ST provider that supports highest level of customizability for your trading style

2. Know the nuances of the BT and ST platforms you are using

3. Pick data structure, fill model, and other settings that most closely match your trading strategy

4. Assure consistency of data, order processing, and fill model assumptions between BT and ST platforms vis a vis live trading

5. Use level 2 tick data in ST

6. Strategies with longer trading horizons and higher expected alpha will have more accurate BT and ST results vis a vis live trading

7. Understand the specifics of the market structure of the products you are trading (stocks vs futures vs FX vs options)

8. Stick to low turnover strategies, liquid trading universe, small order sizes, and be conservative