Garbage In, Garbage Out: The Role of Trading Strategy in Venue Analysis

Ian Domowitz, ITG

28 August 2014

Consideration of trading strategy is an essential component in assessing venue performance, according to ITG research.

Kristi Reitnauer, Assistant Vice President, Analytics Research, and Colleen Ruane, Director, Analytics

Research, contributed to this report.

Venue analysis is all the rage these days, but an old theme. Nasdaq and the NYSE once fought to advertise the

relative execution quality on their venues, motivating academic work purporting to make the comparison. With

the arrival of electronic trading venues, initial efforts concentrated on explaining the mechanics of different

venues, followed by empirical studies, which attempted to contrast execution on electronic and floor systems.

The analogue in the past few months has been the race to publish Reg ATS filings in the interest of

transparency. Venue quality reporting via regulatory mandate appeared with Reg NMS. Closely following, the

interaction of dark pools with routing schemes generated work on information leakage. Comparisons of lit and

dark markets began to appear. In the midst of all this activity, sponsors of alternative execution venues began

to publish material on the relative benefits of their own dark pools. Flash forward to present day, and the furor

surrounding the recent Michael Lewis book has generated collective amnesia with respect to evolution of our

knowledge of venues over the past 20 years.

The focus is on microseconds. We reject that line of inquiry here, and we are not alone. Firms such as Pragma,

whose business is arguably in tiny fractions of a second, note that drawing conclusions from microsecond

snaps of fills is a fruitless undertaking. After investigating several popular methods of venue analysis, Pragma

concludes that such methods do not help understanding or improve performance; they turn their attention

instead to venue fee structures and the patterns of routing behavior created by them.

Venue performance cannot be separated from trading strategy, market conditions, or even constraints on

traders’ workflows. We have made this point before, in the context of comparisons of algorithmic trading

strategies. More broadly, venue analysis is most useful when viewed in the larger context of overall

performance. Understood in that light, a better grasp of routing practices and performance permits traders to

use the full suite of electronic tools at their disposal effectively, as well as generating more useful

conversations with their brokers.

We address a single question in this paper: Is consideration of trading strategy an essential component in

assessing venue performance? Before this issue is settled empirically, it is impossible to construct comparisons

of individual venues, or even contrast dark versus lit markets in the aggregate.

The answer to the question is “yes.” Even examination of the distribution of trading strategies across venue

types suggests that different venues are used to implement disparate strategy choices. We begin with some

comments on our data and a few definitions, which are required to make the analysis transparent.

Data and Terminology

A blizzard of numbers and charts is avoided by sharply limiting the universe of data for this analysis. Trading

activity is for the U.S. only, and restricted to large-cap stocks. High-frequency trading is concentrated in this

capitalization subset. All activity took place over the 2013 calendar year. Small orders are analyzed, up to 1

percent of median daily volume. This last restriction is not terribly binding, since the vast majority of orders

submitted for algorithmic trading fall into this category. The data are not limited to ITG executions; all buy-

side participants represented in the sample trade with a variety of brokers. This means, in particular, that we

study order routing and venues in the large, and the results do not reflect directly on ITG’s methodology.

There sometimes is confusion over terminology, and we want to be as clear in this regard as we are with the

data. An algorithm or trading strategy determines how many shares to be traded at any particular point in time.

The router takes these instructions and determines venues for the execution. A trade or execution is a single

fill, and is typically only a partial fill of the order. Venues are characterized as being either lit or dark. In terms

of pricing models, we restrict most of the analysis to maker/taker venues. An inverted model is taker/maker,

and comprises about 5 percent of the data. We present differences between the two as suits the context; if a

chart does not differentiate between the two, it is based on maker/taker only.

[Related: “Banning Payment for Order Flow Would ‘Magnify Conflicts of Interest’: Larry Tabb”]

Algorithms are aggregated across brokers into several categories. We refer to scheduled strategies, which

include VWAP, TWAP, and Participation strategies. Dark denotes a liquidity-seeking strategy concentrating

on dark pools, while opportunistic is general liquidity-seeking in nature. “IS” is shorthand for implementation

shortfall, creating execution patterns based on cost and risk minimization. Non-algo in the charts is generally

desk trading, while Other is dominated by direct market access and includes some pairs trading.

Some Venue Characteristics by Strategy

Trading strategies can affect venue analysis in several ways, given the profile of fills flowing back to the

algorithm from routing information, and disparities in how strategies cycle child orders into the markets.

Routers also may be tuned to favor certain venues depending on strategy, a more direct approach. Venue types

are characterized by different mixes of strategies which feed them.

Some disparities in venues across strategies are simply intuitive. Dark pool trading is dominated by dark

aggregation strategies, no surprise there. Scheduled strategies favor lit markets; but even within that category,

the taker/maker model exhibits proportionately higher activity, possibly due to the interaction between strategy

and fee structure. Implementation shortfall algorithms make up about a quarter of lit market activity, but

account for less than 10 percent of what we see in dark pools. Similarly, a small fraction of DMA orders hit the

dark pools directly, compared to the lit markets.

Fill size typically is considered a performance metric, but may also be a characteristic stemming from

segmentation of participant types in any given venue. We cannot observe segmentation directly. Fill size is

traceable to the nature of strategies, however, given the interaction of trade size with sizes determined by the

strategy itself. The disparity across venue types is illustrated in Figure 2, below.

At the lower levels of the percentage of average daily volume, there are similarities across scheduled strategies

and implementation shortfall, but also differences. Opportunistic trading may look similar to dark aggregation,

venue type by venue type, but it is quite different from the other two categories in the chart.

Although it is possible to compare these charts line by line, that is not the goal of the exercise. Rather, the two

charts, representing two strategy choices, should be viewed as heat maps, or mosaics, with respect to

underlying activity. In that sense, the two graphs look very different. Strategy type clearly matters with respect

to fill size, regardless of whether it is viewed as a performance metric or as simply a characteristic of venues.

Common Performance Metrics

If venue characteristics vary across strategies, performance comparisons also must be affected. Even the

simplest performance statistic, average spread capture, varies across venues by strategy.

Differences in spread capture for maker/taker markets vary by as much as 16 percent, for example. That figure

rises to 45 percent for the taker/maker paradigm. Dark market spread capture can vary by 46 percent across

strategy types. In terms of the “classic” measure of trading performance, these differences translate into

variation across venues, but traceable to trading strategies, as opposed to distinct properties of the pools

themselves. Figure 5, below, contains results on implementation shortfall cost at the order level.

We expand the strategy set in this chart to include DMA activity (Other) and trading desk activity (Non-Algo).

The difference between these categories for the maker/taker venues is as much as 80 percent. Even a

comparison between implementation shortfall and opportunistic strategies for lit markets differs by 29 percent.

Dark aggregation juxtaposed with scheduled strategies yields a 49 percent difference in cost for the same

venue type. Any comparison of dark pools to lit markets fails, if strategy is not taken into account.

Given the idiosyncratic nature of time stamps in trading systems, calculation of implementation shortfall cost

is typically possible only when a broker strategy is considered. For this reason, many have resorted to

reversion metrics, which depend only on a snapshot of price, at and after, an individual fill. We now turn to the

effect of strategy on such analysis.

Reversion Metrics

Reversion measures differ by the time after the fill. The idea is that if a great deal of reversion is exhibited in

price over very short time horizons, then the fill is bad relative to what might have happened in the absence of

toxicity in a venue. We illustrate 1-second and 5-second timeframes in Figure 6, below.

The differences across strategies, within and between types of venue, are clearly evident. Within maker/taker

markets, at the 5-second level, the variation between scheduled and opportunistic strategies is more than 60

percent. For 1 second reversion, the difference in cost for dark pools across strategy types ranges up to 73

percent. Most of such deviations jump right off the chart. Disaggregated results appear in Figure 7, below. We

limit these charts to 1-second reversion, but the 5-second graphs tell the same story.

Viewed as heat maps, the pictures are obviously different. Within each strategy category, one might now

attempt a comparison across individual venues. We give a flavor of this approach through an examination of lit

and dark markets along the dimension of price reversion in Figure 8, below.

Comparisons across charts illustrate performance in terms of price reversion between dark and lit markets.

Within each chart, individual venues may be compared, controlling for strategy. We are not at the point of

“league tables,” however. Controlling for market conditions should be a part of such an exercise. Nevertheless,

examination of results for implementation shortfall strategies yields a couple of discussion points. The first

point is the appropriate comparison, which depends upon one’s perspective. If the issue at hand is the

proportion of activity which exhibits little reversion, then the correct comparison within and across charts

might be a range around zero, between -0.5 bps and 0.5 bps. If the probability of a bad outcome is of interest,

the tails of the distribution matter, and the correct item of attention might be the unlimited range of less than -1

bps.

On the basis of ‘little reversion,’ and for implementation shortfall strategies only, the New York Stock

Exchange exhibits only about 13 percent of activity within the range around zero. The range between best and

worst performers is rather narrow in the lit markets, however. BATS, for example, exhibits about 16 percent of

activity around zero reversion. There are more significant differences in the extreme tail behavior. The

difference between best and worst performance across lit venues with respect to the probability of a bad

outcome is roughly 44 percent.

Comparisons across dark pools exhibit more variation in the distribution of outcomes. The difference between

pools with respect to straddling zero reversion is as high as 65 percent. For ITG’s POSIT®, 43 percent of fills

are close to zero reversion, and few venues fall below 30 percent in that category. On that basis, dark pools,

controlling for this particular strategy type, dominate lit markets in terms of performance.

The Answer to the Question: A Last Look

A complete set of comparisons across lit venues, dark pools, and comparisons between the two types, is

beyond the simple goal of this paper; we will return to that exercise in the near future. We emphasize our main

point one more time before closing, however. Figure 9, below, illustrates dark pool performance in reversion

terms, now for dark aggregator strategies only.

This chart should be compared to that of dark pool reversion in Figure 8. The percentage of fills for which

there is reversion around zero is now much higher than observed for implementation shortfall strategies. The

range for the latter is roughly between 15 percent and 45 percent. The use of a dark algorithm, which by its

nature is tuned to dark pools, yields a range between 25 percent and more than 60 percent; in fact, for nine of

10 pools, the bottom of the range is above 40 percent. The percentage of zero reversion fills for ITG’s

POSIT®, for example, is 50 percent higher for dark aggregation strategies, as opposed to implementation

shortfall.

The difference in comparisons, and the potential advantage to using an algorithm tuned to the venue type, is

also evident in an examination of the tails of the price reversion distribution. We noted large percentages of poor outcomes, in the case of the implementation shortfall strategies. The tail of the distribution for reversion

in the dark, using dark aggregation, is much smaller, overall, and for nine of the 10 individual venues.

Performance around zero reversion improves, and the probability of high reversion numbers falls.

The answer to our original question is now easily stated: Consideration of trading strategy is an essential

component in assessing venue performance. This is particularly true when attempting to compare and contrast

different market structures, such as dark pools versus lit markets. It also is important in assessing venue quality

on a disaggregated basis, within market structure categories. The results suggest that proper “tuning” of routing

functionality to strategy may improve performance as well. A proper treatment of this suggestion is a natural

next, and constructive, step in venue research.

This research originally appeared on ITG’s website.

© 2009-2014 by The TABB Group, LLC and contributors | info@tabbgroup.com | +1-646-722-7800