Building a System

BUILDING A SYSTEM John Jonelis

Presented to the Chicago Area MetaStock Group May and June 2006 Copyright 2006 John Jonelis, Revised 6/14/06

SUMMARY

These notes describe two workshops on building systematic trading models. Subjects covered are as follows:

Comments

Backtesting 101

Building an indicator

Building, testing, optimizing and proving a system

Testing across multiple markets

Backtesting 201

Testing with limit orders

Accounting for the overnight gap

Backtesting 301

Extending MetaStock by use of Excel

Handling optimized variables and documenting research

Applying comparative statistics

Applying a Monte Carlo simulation

Websites, Acknowledgements, Disclaimer

COMMENTS

I was given a heavy dose of Efficient Market Theory while at Northwestern in the 80s getting a Masters somewhat

late in life. The concept of efficient markets didnt ring true to me then, and I dont believe in it now. However, a

lingering respect remains for the complexity of finance, the time value of money, the intuitive use of statistics as a

blunt instrument, and the power of collaborative effort. Ive left fifty behind and at last count havewith the help

of unselfish collaboratorswritten hundreds of indicators and trading systems. To date, there are only a few that I

actually trade. It seems that it is not easy to find and exploit market inefficiencies on a consistent basis. But permit

me the opinion that what appears as noise to an investor may be signal to a trader.

It would be presumptuous and absurd to put forward a best method to such a complex subject as system

development. Thinking people disagree, and many have found success by other means. The following topics are

based on my personal philosophy. It is a structured approach, and goes as follows:

Test Using Broad Averages

It is difficult to find truth in the midst of turbulence. A close comparison of a number of individual trading

instruments will reveal that each has its own unique signature, one wildly different than the next. Price action is

subject to unexpected and drastic moves brought on by obscure information, unique to each particular security.

In contrast to an individual instrument, an index is more stable and predictable, because it represents an aggregate of

many parts. Test using the averages. Find out something about the market and work from that point. Test

combinations of indices that have disparate characteristicsparticularly ones of differing persistence and volatility.

Select among the models that work beston averageacross a set of such markets.

Test Across Large Sets of Data

Test against as much data as possible. The more data in the test, the more market conditions encountered. If the

strategy is multi-day, gather data from several markets and test across decades of it. Select between the methods

that work in most every applicable situation. Markets change, but prices still rise and fall. Past infers future

because tomorrows data likely bears some resemblance to data in the past.

Along these lines, our research group has created artificial data that mimics the characteristics of individual

marketscomplete with volume. The data exhibits fat tails, as do distributions of real markets. The idea is to test

across 1000 years of Dow, 1000 years of Nasdaq, and so on. Our data is not yet commercially available, but I

recently came across a book that may help those interested in doing something similar: THE (MIS) BEHAVIOR

OF THE MARKETS by Benoit Mandelbrot (ISBN 0-465-04355-0). Hes the inventor of fractal geometry.

BUILDING A SYSTEM - JONELIS - 2

Please notethe typical stylized variety of random data is of limited use. By definition, it conforms to the standard

distribution, and therefore cannot adequately describe the risks of real markets. I urge you to go beyond the

common academic approach.

A note on testing the equity index futures: Start by testing against index or cash market data. This may seem

counterintuitive, but it helps avoid a peculiar problem. Continuous futures contracts plot false price swings due to

the premium at contract expiration and rollover. This calls into question the validity of a test. Another advantage

significantly more index data is available than futures data, and quantity is the name of the game. Of course, it is

also true that any given futures contract is likely more volatile than the related index. But test on the index first,

then verify the completed model against the futures.

If the strategy is sufficiently short term, dont use continuous contracts at all. Test against individual contracts.

Because of its ability to simultaneously test multiple markets, MetaStock is capable of testing Futures on a contract-

by-contract basis.

Take Advantage of the Law of Large Numbers

The markets provide the opportunity, within certain constraints, to set up and run your own gameand people will

actually belly up to your table. You can open the table at your discretion and close the game when you choose.

Your table is a computer programyour game, a set of tested strategies. Your rules are the system components, as

well as the money management and execution practices you have determined to use. Create your own game and

play by your own rules. Be consistent and honor your rules.

Insurance companies, brokers, banks, and casinos each play their own game, and honor their rules. They make their

profits from the average expected return over a large number of events. A trader can do the same. Once a real

market advantage is identified, repeat the trade as many times as possible. If the model is valid, then the more

reps achieved, the closer the system can be expected to perform as tested.

How many trades are needed to achieve the expected result? The formula for confidence level is expressed in

Microsoft Excel as 1-(1/sqrt(n)), where n is the number of repetitions. Turns out, it takes 400 trades to achieve

a 95% confidence levelthe standard used in medical science. An intra-day trader may commonly generate this

volume of activity in any given year, but most multi-day traders will be hard-pressed to do so. Therefore, be

prepared for drawdowns in trading capitalexpect both good years and bad. On the other hand, it takes only 100

trades to gain a 90% confidence level. One hundred multi-day trades per year is a not an unreasonable goal, given a

suite of short-term systems all using different logic, traded on a number of relatively uncorrelated markets.

In system trading, total return is additive. Dont throw away a good system, simply because it does not yield

impressive annual returns. Consider each model as one part of a larger machine. If your systems are short-term and

valid, then the more systems you deploy and the more markets in which you participate, the larger the return.

Consider a set of systems, each yielding an average return-per-trade (or expectation) of one half of one percent per

trade. Lets say that the set generates 30 trades in one market, and in another relatively uncorrelated market, it

generates an additional 20 trades, and another, 10. Thats 60 total trades, yielding an average total annual return of

30%. Meanwhile, there is an entire industry of money managers struggling to outperform the standard benchmark

the S&P 500, which yields an average of 7-10%, depending on what period is measured.

Diversification is the most common way that investors take advantage of the law of large numbers. Diversify across

markets, methods, and time. Trade a number of systems, each based on different logic. Trade at least three

somewhat uncorrelated markets. Choose markets that zig when the others zag. Choose markets that respond

differently to events. Variety smoothes the overall equity curve, and to the extent that trades overlap, a degree of

hedging may be enjoyed. Such diversity provides a measure of protection against catastrophe.

Undisciplined leverage negates the law of large numbers, because too much risk is placed on individual trades, and

too much capital is tied up in a few trades. Be a plugger, not a plunger. As in baseball, a strong batting average

is achieved by emphasizing singles and doubles and letting the occasional home runs come as unexpected delights.

Try to hit as many singles as possible.


Measure and Control Risk

Use Fixed Fractional Money Management. It is both highly effective and easy to implement. The method defines

risk as the percent of trading capital that is acceptable to lose if a trade stops out. Simply determine an appropriate

percent to risk on each trade, and then manipulate the size of each trade to achieve the same risk every time. As the

account grows, trade size increases. As the account draws down, trade size diminishes. The percent risked stays the

same. There are two good books on the subject by Thomas StridsmanTRADING SYSTEMS AND MONEY

MANAGEMENT (ISBN 0-07-140019-2) and TRADING SYSTEMS THAT WORK (ISBN 0-07-135980-X).

One can be more precise, but as a general rule, never allow open-trade risk to exceed 2% of the trading account on

any one trade or combination of trades in correlated markets. In other words, dont risk 2% of your account on the

S&P 500 and 2% on the Nasdaq at the same time. Even though these two markets are different in volatility and

persistence, both will tank simultaneously given a major catastrophe. If you adhere strictly to this rule, even a string

of twenty consecutive losses will not cause your account to blow up.

Use Monte Carlo analysis to measure the risks of any given system or set of systems. For help in using this tool

without a degree in math, see my lead article, ALTERNATE HISTORIES in the JOURNAL OF TECHNICAL

ANALYSIS (2005, Issue 63, Market Technicians Association). That paper started as a presentation to this group.

Beware of Circular Thinking, Coding Errors, Clairvoyance, Compexity and Curve Fitting

Circular ThinkingSolitude permits introspection but does not breed ideas. Left alone, one tends to think in

circles. Ideas are neededlots of ideasoff-the-wall ideas. I highly recommend collaborating with others. Ideally

one would seek the following attributes: 1.) Creative thinkers with completely different modes of thought. 2.) The

ability to work as a unitthrough brainstorming, division of work, and mutual kindness. 3.) The integrity to respect

the intellectual property rights of the team. I realize that such a list may sound hard to fill. I have been fortunate.

My own research group consists of an individual with a gift for teaching and spotting errors, a statistician, an

experienced technical trader, and me. None of us are programmerswe stretch ourselves to accomplish the task. It

has taken three years to develop our current crop of systems. But I have never before been involved in a project that

provided so much in the form of creative challenge, reward, and outright fun.

Coding ErrorsModel building requires programmingat least to a limited extent. Writing code inevitably

yields errors. Just as nobody should be expected to proof his own essay, so in like manner, a system developer

should not have to proof his own concepts and code. Its a formula for frustration and stagnation.

ClairvoyanceStrange as it may sound, this is a constant problem facing the system developer. Simply stated,

clairvoyance is allowing a system to take advantage of information that would not yet be available in a real trading

environment. Ill give just a few examples:

Indicators that calculate at the Close, while the system executes intra-day

Execution at the High or Low of a bar

Mismatches between the price at time of trigger and the price specified in the stop-limit field

Whenever test results look too good to be true, the system should be tested for clairvoyance. If you have a real-time

platform, there is an easy way to check for this problem. Simply plot the indicators or signals on a 1-minute real-

time chart of a highly volatile and liquid market, and watch for changes in the signal history.

ComplexityThe greater the number of conditions, the higher the degrees of freedom. In other words, each

added condition tends to randomize test results and more iterations are needed to maintain the same statistical

significance. Taken from a practical perspective, the real goal is to generate a lot of trades per year. Too many

entry conditions reduce the number of trades, because each new condition makes the trigger more exclusive. On the

other hand, the more exit conditions, the sooner the trade is terminated, potentially increasing the number of trades.

I favor simple entries and complex exits.

Curve FittingThis is probably the number-one plague on the population of system developers. As an analogy,

the higher order a regression, the closer it fits an existing data set. But often, the higher order the regression, the less

its predictive value. An illustration: Sometimes a tightly fit regression will predict one outcome with remarkable

accuracy, and then on the next several iterations, will choose outcomes that are wildly wrong. On the other hand, a

simpler regression may never provide accuracy, but may tend to land in the general vicinity on a more consistent

basis. So it is in building systems. Often, the closer a model fits a data set, the less predictive it will be. Since the


characteristics of any future data stream are known only by inference and subject to abrupt change, I recommend the

value of the blunt instrument. The first step is to recognize that, to one degree or another, all trading methods are

curve-fit. The goal is to create one that isnt too tight. A system is called robust, when it works, in general, across

a variety of market conditions. It is unspectacularwhat might be called, good enough.

Steering away from over-fitting the curve is as much an art as a science. Permit me to point out three avoidance

techniques:

The MetaStock handbook recommends testing across the oldest half of a data set, then validating across

the remainder of the data. This is a very simple example of out of sample testing.

In my last presentation on this topic, I described an established and elaborate method known as the Walk

Forward Test, complete with descriptive chart. Basically the Walk Forward involves optimizing a system

across a defined chunk of data, then testing out-of-sample on the next chunk. If the out-of-sample test

verifies the model, then the process is rolled forward and repeated. Then again, and so forth. Using this

method, a system can be kept in phase with changing market conditions on an ongoing basis. Walk

Forward Testing is, I believe, still the industry standard. But it is elaborate and time consuming and there

is some question as to whether or not it is the best solution to the problem.

The method that I currently use is simple. Apply lots and lots of data, and to use all of itright up to the

most recent sets. For out-of-sample testing, I simply put aside the data from one of the markets that I hope

to trade. Many will disagree, but I believe that this provides a rigorous test. Permit me point out the

advantages of this approach. It relies on three simple ideas.

New data is important because the system must work in current market conditions.

Old data is important because market conditions change.

A system should perform well across both old and new data.

Most of my methods take the Long side. Starting in March of 2000, the US equity markets went into a

protracted bear run. For the system developer, this is an unusual opportunity. If an idea works during the

bullish phase, which represents the bulk of the dataand also during the bearish phase, which is the recent

datathen it is likely a robust solution.

But let each researcher choose the method that seems appropriate.

Design a System that Suits You

A system is useless if it does not match your personality. Do you want to be right most of the time, or do you prefer

to avoid adverse price moves? At the extremes, one approach might provide many small winning trades at the

expense of a few large losses, while the other might result in numerous small losses and only a few big gains. In

general, the degree of probability and the reward to risk ratio display an inverse relationship. You must find the

combination that suits your personality or you will be unhappy with the result. There are many other issues to

consider. For example, will you ignore the trade during the day, or will you watch it in real-time? What time frame

do you prefer? Are you an introspective or a reflexive thinker? Are you a thrill-seeker or risk averse?

There are purely discretionary traders, who do their analysis on a case-by-case basis. There are purely non-

discretionary traders, who make all decisions in advance, trading purely by rules. In truth, most fall somewhere in-

between. Who is right? I have to admit that I know traders who make money in many different ways and I cannot

find cause to condemn any of the approaches. I think the question boils down to game playing. Do you like a good

hand of poker, or do you prefer chess? If you like a fast-paced round of cards, you may make a good discretionary

trader. But if you prefer to sit back and ponder a chessboard, consider the advantages of a non-discretionary system.

A systems approach allows you to calmly make your decisions in an abstract environment and apply statistical tests.

Execution can be purely mechanical, with workdays spent creating more strategies. But remember, to receive the

advantage-as-tested, one must follow the system and not fiddle with it on the fly.

To read about system development in storytelling form, I recommend the wonderful book, THE PREDICTORS by

Thomas A. Bass (ISBN 0-8050-5757-9). Model builders will find this book a joy. I found in it a much-needed

source of encouragement.

Now lets move on to a practical demonstration. We will conceive the most basic strategy possible, build a simple

model, and then test it, and determine whether or not we want to trade it.


BACKTESTING 101

Following are illustrated demonstrations of certain basic functions in Equis MetaStock Professional version 9.

These functions are foundational to those wishing to take full advantage of the MetaStock platform. Please make

sure you have a firm grasp on the 101 materials before moving to 201. This demonstration is designed such that the

reader can practice the exercises by following along. Bear in mind that what is shown is only one of many ways to

use this large and complex software package. It is assumed that the reader has a basic familiarity with the program.

These procedures do not replace the official manual. Rather, it is hoped that this demonstration will serve to

shorten the learning curve. If errors are encountered, please refer to the manual, and contact the group so that

corrections can be made.

Create an Indicator

Before we move on to systems, it is important to know how to write custom indicators. We will create a Displaced

Linear Regression, called DLR Rapid. Click the F(x) icon as shown. This is brings up the Indicator Builder.

Click New. The Indicator Editor will appear (see background box above).

Type in some meaningful name. (Ill call it DLR Rapid)

Click Functions and a list of indicators will appear (see foreground box above).

Select Linear Regression Indicator.

Check Paste Arguments in the lower left corner. Click OK.

Your indicator will appear in the editor, (see below):

Function with Arguments Displayed Insert values and apply Ref() function

A linear regression function comes up with instructions pasted in the window (above left). Simply replace the text

with the desired values, as follows: The values are LinearReg(L,10,S,10). A high degree of smoothing has been

chosen so that the indicator will show a definite direction and not jiggle around. This is an important consideration

when using an indicator in a systemit helps prevent a series of quick reversals. One needs to be definite. Next,

type-in a Reference function to displace the indicator 3 bars to the right, as in Ref(LinearReg(L,10,S,10),-3). Click

OK. The indicator will appear in the Drag and Drop box. Drag the indicator onto the price window.

Click F(x) at the top of the chart display. This will bring up the Indicator Builder.


The above chart shows the entire display with the custom indicators plotted. Volume is shown in the lower window.

To build a system, our first step is to visualize the strategy on the price chart, using the Expert Advisor.

WarningWe are about to build a simple demonstration system that I call HEART ATTACK. The idea is to

highlight the basic concepts, and not to get bogged-down in programming. This system is intended purely for

demonstration purposes. Use it as a training tool. Please do not trade it.

Create another indicator:

Scroll down to PS StochRSI. Click Copy.

Name the copy. (Ill call it 8 StochRSI 10)

Click Edit and remove the last two items of code.

20;

80 (The 20 and 80 are horizontal reference lines, which

prevent the function from being used in a backtest.)

The PS StochRSI is part of the Performance Systems

Plus plug-in. MetaStock has granted permission to

post the StochRSI code in this paper. Our change

results in the following function:

(Sum(RSI(10)-LLV(RSI(10),10),3)

/Sum(HHV(RSI(10),10)-LLV(RSI(10),10),3))*100

NoteI have chosen this indicator for our workshop somewhat at random, but also because it oscillates to extremes, even during flat markets. I hope that it yields a large number of signals. I have not worked with it in the past, and look forward to seeing how it behaves.

Plot the new indicator in a separate window on the

chart, as shown below:


Create an Expert

MetaStock is extremely visual. Type a concept in code and the result appears on a chart. That means that concepts

can be written and displayed, much as an artist uses brush and paint. We will begin by creating an Expert.

Click the Man with the Hat icon (top of price display) to bring up the Expert Advisor (background window below).

Entry Symbol

Click the Name tab.

Type in the name, HEART ATTACK 1 BUY

Enter the following defined variable: (Defined

variables are used as a kind of shorthand.)

S1:= Fml(8 StochRSI 10); (note semicolon)

There are three arguments to this entry signal:

Cross(S1,20)function must cross up thru 20%

H>Ref(H,-1)a higher high is also required.

C>5prevents any signals on low priced issues.

Click OK

Simply stated, buy when indicator crosses 20

Exit Symbol

Create another Symbol for the exit.

Click New and create another symbol.

S1 is the same defined variable.

Cross(S1,95) says that S1 must cross 95. The code simply states Sell when the indicator

crosses up through 95. This is our profit target.

Select a graphic and click OK (The symbol graphics can be serious or fanciful,

whichever you prefer.)

Expert Advisor dialogue box (background)

Click New to bring up the Expert Editor (foreground).

Enter the name of the system, HEART ATTACK 1.

Select the Symbols tab and Click New (background).

Click the Graphic tab and choose options as desired

(Select symbol type, color, notation, font, and location).


Bring up Microsoft Notepad. Paste-in the code. Notepad is a good place to write and store code for later use in

various places within MetaStock. Below is the code for our system, as it might appear in Notepad:

Bring up a chart and right-click in the price window. Choose Expert Advisor, then Attach. Scroll to the Expert

that was just written and click OK. The symbols and highlights should appear on the chart.

The figure above shows the HEART ATTACK 1 system plotted on a chart, with the individual trades boxed-in by

hand for illustration. This is a visualization of the model. Note the way the strategy catches intermediate price

swings and exits on strength. Note also the outsized loss on the left-most trade. From a brief look at this small data

set, it appears that the system is acting as expected, but that more work will be required to make it tradable.

BUY--- S1:=Fml(8 StochRSI 10); Cross(S1,20) AND H>Ref(H,-1) AND C>5

SELL--- S1:=Fml(8 StochRSI 10); Cross(S1,95)

Entry HighlightClick the Highlights tab. Click

New. Copy exactly the same entry code used for the

entry symbol from Notepad. This paints a color over

a bar, in lieu of a symbol. Choose a color. Click OK.

Exit HighlightCreate another highlight for the exit, again

using the exact code as the exit symbol. Choose a color. Click

OK. MetaStock cannot display two symbols on the same bar,

so Highlights will be important later in the procedure


Create a System with an optimization variable

Click on the $ icon at the top of the MetaStock display. The System Tester will come up.

Click New System in at top left.

The System Editor will come up.

Name the system and enter any

notes.

Order BiasClick the Long radio

button. If a Long and Short signal

both trigger at once, the system

will select the Long order. Portfolio BiasSelect Multiple

Position LimitCheck the box

and enter the number 1.

Click the Buy Order tab. Paste-in the Buy code from Notepad.

Order TypeSelect Market

Limit or Stop PriceLeave blank (covered in next workshop)

Entry SizeSelect Use Default Size

ExpirationSelect Good Until Cancelled

Strategic DelaySet these all at 0ALWAYS ZERO.

Click the Sell Order tab. Paste in the Sell code from Notepad.

NOTEReplace 95 with OPT1 as in Cross(S1,OPT1)

All other settings are the same as shown above.

Click the Optimization tab. Click Edit.

The Variable Properties box will come up.

Give the variable a name( Ive called it upper line)

Enter the minimum and maximum values(Try 75 to 95)

Enter the steps between values(5 is enough granularity.)

Click OK

Click OK again to close the System Editor.

The HEART ATTACK 1 System has been created.

It may seem counterintuitive, but this combination will instruct MetaStock to trade only ONE position at a time, and keep the position open until an exit signal is generated. Entry signals triggered while a trade is open will be ignored. Opposing signals will not cause a stop-and-reverse.


Test the System

Now lets test this strategy across a diverse set of equity indices.

Click on the $ icon to bring up the System Tester. Scroll to the system,

HEART ATTACK 1 and highlight.

Click New Simulation (bottom of

background window). The Select

Systems dialogue box will appear.

Verify that the correct System is

selected and click Next. The Select

Securities dialogue box will appear.

NOTEIt is recommended that all

tests use Local Data stored on the

hard drive, as opposed to streaming

Real-Time data.

Click Next to bring up the Select

Securities box.

Click Add Securities and browse

to the file of choice. Select a

security and click Open.

Highlight the security and click

Dates. Enter the date range. (I will

use 7/22/1965 to 7/22/2005.) Click

OK and enter another security.

When all the securities are entered,

click Next to bring up System

Testing Options.

For demonstration purposes, we will restrict this test to readily available data. (Ive entered the S&P 500 and Dow Industrials from 1965 to 2005, using data from TC-2000, which I have rounded off to the nearest tick size.) That will test the strategy across a variety of conditionsfrom range-bound to strongly bullish to strongly bearishfrom persistent to choppyfrom sleepy to volatile. Ill add the Nasdaq (from 1990 to 2005) to make the test more rigorous. The Nasdaq is more volatile than the other markets, and the period chosen is at least one-third bear market. This should prove a difficult test.

MetaStock is capable of testing

multiple systems against multiple

securities. Because of this, we can

obtain the average result of all

markets tested. This is crucial to

creating a robust system.


Click OK, then START. The simulation will run.

In System Testing Options, set Initial Equity to 100,000.

This will make all the performance numbers easy to

interpret.

Set Default Size to % of Available Equity, enter 100.

This will force the size of each trade to be equal to the entire

account. This way, disparate securities will be readily

compared on equal footing, and all our tests will run to full

term..

Use other settings as shown. For this test, we will

uncheck the Quick Test box, so that the system will keep a

log of each trade. We will need that for our proof at the end

of the procedure.

Click More to bring up Broker Options.

MetaStock does a good job of making a simulation feel like real-life trading. But its important to understand that it is not the same as real-life trading. There is nobody on the other side of the trade reacting to what you are doing. In the simulation world, Market Orders are just as good as Limit Orders. All trades are filled at the correct price, every security has unlimited liquidity, and position size is unbounded. In the simulation world, futures have no leverage and all that is known is the price on the chart. Certain matters need to be viewed as if in a mirror.

In the Broker Options tab, set Margin and Money Market

interest rates both to zero. We are testing a system, not an

estate plan.

Set Margin requirements as shown: Long Initial = 100,

Long Maintenance = 0, Short Initial = 200, Short

Maintenance = 101. With these settings, each trade will be

sized at 100% of remaining equity, and positions will be

unleveragedmargin will be used merely as overdraft

protection. Each test will run to full term, even if it loses

money. Also of importancevalid signals will not be

refused for brokerage reasons.

CommissionsInclude both commissions and slippage in

these windows. Select Percent of Transaction (%). To test

futures, calculate your actual commission costs and add the

cost of at least one tick to both sides of the trade to simulate

slippage, then convert to a percentage and round up. To test

stocks, either calculate a percent, or select Points per Unit

($) and enter the actual commission per share plus slippage

of from 1-5 cents per share for both sides of the trade.

In the Trade Execution tab, UNCHECK Realistic Market

Prices, and set Delay order opening to ZERO. All sorts of

unexpected problems occur when testing using a delay.

Program your order delays into your code.

Set Buy, Sell, Short, & Cover Prices to Close or Open.

Keep in mind that most indicators calculate at the close.

(Please do not use High or Low.) These will be your

defaults. A limit price (see Section 201) will override the

default, giving you a lot of flexibility, as will be seen later.

Do not enter slippage under Slippage. This creates

unexpected order problems. Add slippage to Commissions

as mentioned above. That way, your execution price will be

precise, and your slippage will be booked against your

account as a line-item entry.


When finished, click View Results. Because we used an OPT variable, a summary page is generated comparing the

net results, using each value. In my opinion, this page gives the most valuable information in the entire package. It

shows the AVERAGE RESULTS ACROSS ALL MARKETS TESTED AS AN AGGREGATE. Even when not

optimizing for a variable, a dummy OPT variable can be created so that this page will be generated.

For now, without getting complicated about it, we will choose the highest return, which turns out to be a StochRSI

value of 95. Note above that an average of 229 trades per market were generated. A quick calculation will reveal

that 72% of them are winning tradesa spectacular result. However, there is a glaring defect in this solution. Can

you detect it? (We will address that question in Section 201.)

Double Click on the line showing OPT1 as 95. Individual results for each market will appear, as follows:

Each market is shown at the value of 95. Why did the Nasdaq do so poorly? Actually, it didnt. Its test merely

contains fewer years of data. It is important not to attach too much importance to the height of the bars. But since

the Nasdaq is our most volatile market, lets take a closer look at it.

Double click on NDX, and a number of reports will come up.


Reports for NDX

Click on the Equity tab. A cumulative equity line is drawn for the NDX. Not a pretty sight. As it stands, I would

not want to trade this system.

Select the Positions tab (see below). This is a record of every trade, and plots the frequency of each trade result

on a percentage basis. The more heavily weighted to the right of zero, the more robust the solution. This chart

highlights the strength of this particular strategy.


Click the Summary tab. The page includes analysis of various types.

Some of the interesting numbers include annualized performance, comparison to buy-and-hold, highest and lowest

profit and loss, most consecutive losses, maximum position excursion, highest open-trade drawdown, highest

closed-trade drawdown, Maximum, minimum, and average trade length. There is a wealth of information on this

page, but again, these numbers represent only one security tested on one data set. We will analyze of aggregate

results in Section 301.


Click the System tab. All the system details are recorded here. This is global informationidentical for every

OPT variable, security, and data set used in the test.

This page includes everything needed to reproduce the test, including trigger code, brokerage settings, and limit

price formulas. (Limit prices are covered in Section 201). Code can be copied directly from this page and pasted

into Notepad or Excel (see Section 301). Next, click Plot to Chart (lower right corner).


Click Plot to Chart, and CLOSE THE SYSTEM TESTER. The trades are plotted directly on the chart of the

NDX. Now bring up the Expert that we wrote, using the Highlights (not the Symbols) for BUY and SELL. (The

boxes are drawn by hand for clarity.)

Do the colored highlights match what the system as plotted? They do. By holding the cursor over each trade

number, the execution price is displayed, and can be compared to design expectations. This is a proof that the test

performed as expected.

Conclusion to Section 101 Up to this point, we have conceived a strategy, written an indicator, visualized a model, tested it, and proven the test. Next, we will perform intra-day execution on daily bars using limit orders.


BACKTESTING 201

In this section, we will use limit orders to perform intraday trade execution on daily data. Further, we will backtest

in a way that takes into account the overnight gap. Understanding of the 101 material is essential to practicing the

following procedures. The examples are simple, but the concepts may open the door for further exploration.

Create an Adaptive Stop Loss

When testing multiple markets across wide time frames, it is useful to create a stop loss indicator that adapts to the

local volatility of the issue being tested. If the Stop does not adapt, then a different value will be needed for each

security, and the Stop value will have to be adjusted over time. We want one number that works across a broad

selection of data. To make the function adaptive, we will use a multiple of the 20-period Average True Range, or

ATR(20). This will be a Chandelier style stop loss, which hangs from above.

The stop loss function is shown plotted on the chart below. It plots a value J units of ATR below the High on the

day-of-entry, and re-sets at each new entry signal. Plot it on your own chart along with the Expert to verify that it

works as intended. The indicator should appear as shown.

{8 Heart Attack Stop Long} J:=0.6; R:=ATR(20)*J; S1:=Fml("8 StochRSI 10"); Y:=Cross(S1,20) AND H>Ref(H,-1) AND C>5; Stop:=ValueWhen(1,Y,H-R);

Stop

Y is simply a defined variable describing our entry condition.

By including this in the indicator, the stop will re-set each time a

trade is triggered. Because the stop automatically resets, it can be

used in a backtest. Because it is based on the Average True

Range, it will adapt to local conditions. NoteMetaStock treats anything in {brackets} as a remark. In

this case, brackets allow inclusion of the title of the indicator.

We have already covered creating an Indicator. All Indicators,

Experts, and Systems can be written and saved in text form.

Microsoft Notepad gives us a convenient workspace and an

efficient way to catalogue such material.

This is a simple and basic stop loss, which carries with it

a limitation: If a new trade signals before the previous

trade closes and the market is in a downtrend, then it is

possible that the stop will back off. However, this

should not prove a problem, given a J number that is

tight enough. In any event, the intent is to demonstrate

the use of a limit order. Complex stops are outside the

scope of these notes. (For examples of complex stops,

please explore the websites listed at the end of this

paper.)

If one prefers a trailing stop to a floor, simply replace

the ValueWhen() function with HighestSince().

In Section 101, we created a system that resulted in winning trades 72% of

the time, and I pointed out that the solution had a glaring defect. The

Avg. Profit/Avg. Loss or Reward/Risk (R/R) was a dismal 0.59. In

other words, the strategy made (on average) only 59 cents for every dollar

risked. I like to see R/R in the 1.60 to 2.00 range, but Probability and R/R

are typically inversely correlated. Digging deeper, it can be shown that

the maximum open-trade drawdown was 34%. This is essentially

undefined risk, much like buy-and-hold, and does not come under the

category of disciplined trading. Clearly, more work must be done. We

may or may not achieve a tradable system within the limits of these notes,

but armed with the concepts shown, and using individual creativity, it is

hoped that each reader will find a unique solution.

To test for maximum open-trade drawdown (single trade) perform a simple optimization, as follows: In the System, go to the Stops tab, and enter OPT3 in the Maximum Loss function. Click the radio button Percent. Under the Optimizations tab, set a range, such as 1 to 50 in increments of 1. Run a test and scan the results for the point at which the output shows no change. The OPT3 value at that point is the maximum open-trade drawdown. For Heart Attack 1, it turned out to be 34%, which gave rise to its name.

(The J value of 0.6 is chosen at

random. Performance testing will be

done in section 301.)


Test the Stop Loss

Bring up the System Tester and copy the system, Heart Attack 1. Name the copy Heart Attack 2. Change the code

as follows:

Limit Orders

So far, any Sell trigger will execute at the default value, which we have set at the Close of the day. Lets tell

MetaStock to enter at the Close, but execute the Stop on an intraday basis. To do this, we will use a Stop Limit

Order. Click on the Sell tab and select the radio button Stop Limit. Click on the f box to bring up the Formula Editor. Enter the following code:

Accounting for the Gap

What if, during a trade, the market opens below our Stop?

{SELL ORDER} J:=OPT2; R:=Ref(ATR(20),-1)*J; S1:=Fml("8 StochRSI 10"); Y:=Cross(S1,20) AND H>Ref(H,-1) AND C>5; Stop:=ValueWhen(1,Y,H-R); Cross(S1,OPT1) or LRef(H,-1) AND C>5; Stop:=ValueWhen(1,Y,H-R); If(LRef(H,-1) AND C>5; Stop:=ValueWhen(1,Y,H-R); If(L5


Setting the range of the OPT variable

We will perform optimizations in Section 301. For now, set single values for each of OPT1 and OPT2, so that a

summary page will be generated. Click on the Optimizations tab, then EDIT, and set the parameters for OPT2:

Proof the Model

Run the test (with details) and click on one of the data sets.

Click Plot on Chart.

CLOSE THE SYSTEM TESTER.

Close (or reduce in size) the Equity Window (upper window of chart) and zoom in.

Bring up the Expert from Procedure 101. Create a new Highlight, and enter the code from the Sell Order tab:

Activate the existing BUY and the new SELL Highlights, and uncheck all other Highlights and Symbols.

Plot the Stop Loss, and make sure the J value for the Stop function matches the test value of 0.6.

Hold the curser over a few BUY and SELL symbols and read the price logged by the System Tester.

Compare logged entries to those plotted on the chart.

Do the Entries and Exits occur at the Close, as intended? They do.

When a bar crosses the Stop Loss, is the correct intra-day price logged? It is.

When price gaps below the Stop Loss, is the Opening price logged? Yes.

NoteAn optimization value cannot be used in an

Expert. When creating an Expert Highlight,

hard-code the OPT value that matches the value from

the testin our example, 0.6 and 95 (each

shown in bold).

{EXPERTHeart Attack 2 SELL} J:=0.6; R:=Ref(ATR(20),-1)*J; S1:=Fml("8 StochRSI 10"); Y:=Cross(S1,20) AND H>Ref(H,-1) AND C>5; Stop:=ValueWhen(1,Y,H-R); Cross(S1,95) or L


BACKTESTING 301

In this section we will move our data out of MetaStock and into Excel. Once the data is on the spreadsheet, we can

sort it, analyze it, compare it, and store it. Bring up the previous simulation for Heart Attack 1, or run a new one.

Note that column F (6th from the left, above) is a fraction. The secret to unlocking the data is to delete the

denominator. Even if faced with pages and pages of optimization results, all denominators can be deleted

simultaneously. Highlight the column, and use Excels [CNTL H] function.

The next problem is that the original numbers are compounded on a trade-by-trade basis. When using long runs of

data, this magnifies performance to unrealistic heights. Unless you are trying to dazzle unsuspecting customers with

big performance numbers, you really would rather know the average return in any given year, independent of other

years. We will change the compounding to simulate starting with the same account size every year. The change is

accomplished using the Log Normal or LN() function in Excel.

The reader will recognize that many other statistics (and even full-color graphs) can be brought out from the same

original output. My focus is on three numbers:

1.) Avg % Gain Per Trade (A pure comparative number, this is the average of all wins and losses.)

2.) Probability (A measure of how often one can be expected to land on the right side of a trade.)

3.) R/R (A measure of how much, on average, one can expect to profit for every dollar put at risk.)

(During the workshop presentation, I demonstrated a way to describe the aggregate equity curve, using two rough

numbers, as well as some comparisons to buy-and-hold. These have been omitted from the notes for simplicity.)

A MetaStock technical representative once told me that people complain that this page doesnt contain enough information. Actually, it displays the best and most important information put out by the program. It shows the average results of all markets tested. But we need to unlock the numbers.

The optimizations

page (at right) shows

the average

performance of all

markets. (This page

was generated because

we included an OPT

variable.)

Right-click on the

numbers, then Cut-

and-paste into Excel.

At right is an example

of a performance

numbers that can be

gleaned from the

seemingly meager

program output. The

sophistication of the

analysis is up to the individual.


Next, lets perform an optimization of Heart Attack 2, then move it into Excel and sort it. As can be seen, all the

code needed to reproduce the System, Expert, and Indicators can be stored along with the test results in Excel:

At left is an optimization of Heart

Attack 2, testing both variables. As

is shown, R/R can be brought above

1.00, but at a bitter cost to all levels

of performance. Stops of J=2 or

tighter may be seductive, but in this

strategy, they throttle the trade.

Its a good thing we tested before

trading. As it stands, the concept of

limit orders has been demonstrated,

but a tradable system has not been

shown.

We cannot further loosen the stop,

because our simple function may

back off during a bear market. A

complex stop is needed for the

testone that will permit more

room for the trade to breath, without

the danger of backing off. For

instructions on writing complex

stops, see the websites at the end of

these notes.

Lets turn back to Heart Attack 1 and see if we can find a reasonable defined risk level for money management purposes. We will optimize using MetaStocks built-in percent stop. In the System, under the Stops tab, find Maximum Loss. Check Long, select the Percent radio button, and enter OPT3 to test for an end-of-day stop level. Well set our range from 8% to 20% in increments of 1%.

For demonstration purposes,

16% appears to be a sweet spot

on the optimization (see

highlight at right). In actual

practice, this type of stop must

be tested individually for each

market, and continually updated

to account for recent volatility.

As is shown, overall

performance is reduced slightly

from the original version.

(That one had an alarming 34%

open-trade drawdown.) The

percent stop may be

primitive, but it at least allows

us to define our risk, and limit

the size or each trade to control

our maximum loss. Lets take a closer look at Heart Attack 1b. Next we apply the Monte Carlo

simulation.


Monte Carlo Simulation

For a detailed procedure on setting up and reading an MC simulation, see the article ALTERNATE HISTORIES

(reference at end of paper). For our engine, well use Equity Monaco from TickQuest. At date-of-writing, it is a

free download. One additional wrinkle is added for this testeach trade is normalized to the same account size.

Run the System Test againthis time with details. Go to the Positions tabs in the System Test. Cut and paste the data (not the graph) into Excel. The three columns at the right are added to the spreadsheet as follows: EQUITY =IF(B__="Total",I__+H__,"") % CHANGE =IF(B2131="Total",(I2131-I2123)/I2123,"") MC INPUT =IF(B__="Total",J__*$_$_,"")

($_$_ refers to the cell containing initial equity. The titles are for illustration. Do not enter titles.) As can be seen, Position # 262 results in a $5,407 gain, which is a 0.9% percent change in equity, which at this point in the simulation is over $500,000. This is raw MetaStock output, and compounding has not yet been removed. In terms of the initial equity of $100,000, 0.9% is about $920. The $920 is the gain normalized to initial equity. Copy the last column to a text file for use in our MC simulation.

The MC equity plot displays a strongly positive slope and forms a nice distribution around the mean. But of 300 paths, many experience a 50% drawdown, and one outlier leads to ruin. A look at the numbers on the Y-axis reveals that terminal equity is quite widely scatteredfrom about $150,000 to $700,000. This is not the kind of result one wants to see in a systematic trading strategy.


The high variance in terminal equity shows us the nasty side of this system. Of particular concern is the 16% open-

trade-drawdown. It will be difficult to trade the strategy within any reasonable risk profile. This is the kind of

system that entices you with a lot of small gains, and then clobbers you with a huge loss. As it stands, I will not

trade the current versions of Heart Attack. That decision defines reason I perform tests.

I still hold out hope that the adaptive intraday stop will transform Heart Attack 2 into a tradable strategy, and I

leave that to the group. What is needed is a complex stop loss function that includes a latch to define whether or

not a trade is open. (Instructions for writing such a stop are referenced below, but the code is outside the scope of

this workshop.) The basic concepts have been demonstrated, but it is left to the reader to find a personal and unique

solution to the system.

The Position Data is strongly skewed positive. This graph points out the strong points in the model, much like the graph on the MetaStock Positions page (see section 101). There is certain robustness to it. It is clear that this strategy has aspects that show merit.

There are 7 consecutive losses at the 95

th percentile.

(see arrow) At first glance, this seems a rather typical result. But this particular strategy can lose as much as 16% in a single trade. If it were not for the fact that our money management strategy reduces position size as equity draws down, seven consecutive loses could lead to ruin. In actual trading, we would reduce position size to control risk, but that would come at the cost of reduced profits.

Conclusion to Section 301 In this section we moved our test results out of MetaStock and into Excel. We removed compounding from the numbers and applied an array of statistics. We performed a Monte Carlo simulation using Equity Monaco, and analyzed the report. That resulted in the decision not to trade Heart Attack until the risk can be better controlled.


Websites

The following websites have been brought to my attention because of their relation to MetaStock.

I post this list at the request of the group but cannot make any specific recommendations or endorsements:

In the MetaStock world, two names have repeatedly been pointed out to me as highly respected for the quality of

their work. Roy Larsen is one of them. His newsletter is essentially a high-level course in MetaStock programming.

Sample issues are available for download, and the entire set, including back issues, can be purchased in PDF format.

http://www.metastocktips.co.nz/

Jose Silva is the other individual referred to me by so many sources. His site includes many free formula solutions:

http://www.metastocktools.com/

The following newsgroup has some highly sophisticated postings. Roy Larsons Forum.dll and instructions are

available as a free download on this site. To access the materials, first join the Yahoo group. Then go to Files/Equis

Forum DLL and find Forum.dll (rlarsen_99) http://finance.groups.yahoo.com/group/equismetastock/

The MC demonstration was done using the Equity Monaco package, which at time of writing is a free download

from TickQuest. http://tickquest.com/

For a primer on MC, see the pre-print draft of my article for the Journal of Technical Analysis, 2005 Issue 63,

Market Technicians Association, Inc. at my resource site. Go to

http://home.comcast.net/~countertrender/Countertrender.html and select ALTERNATE HISTORIES to download

the PDF file.

For examples of a complex stops, visit Larsen, Silva, or our open forum at http://project301.blogspot.com

Acknowledgements

Special thanks go out to those who have collaborated in my efforts so unselfishly, including Bob Jonelis, David

Jonelis, and Charles Warren. Thanks also go out to Joe Blazie, who has led the Chicago Area MetaStock Group for

years, and also to his counterpart at the Chicago chapter of the Market Technicians Association, Ross Leinweber.

Each has provided a wonderful resource, and both deserve the gratitude of all traders who have been fortunate

enough to participate in their forums.

Credit is given to MetaStock / Equis International and TSA Group. MetaStock, Equis, OptionScope, and

QuoteCenter are registered trademarks of Equis International. Achelis Binary Wave, The DownLoader, Expert

Advisor, The Explorer, Visual Control, and Smart Charts are trademarks of Equis International. Equity Monaco is a

product of TickQuest, Inc. All daily data used in this workshop is from TC-2000, a product of Worden Brothers,

Inc. Excel and NotePad are products of Microsoft Corporation.

Limitations

The comments, procedure, and other information are presented to demonstrate one method of backtesting using the

MetaStock platform. It is hoped that others will add to what has been presented. The material is for educational

purposes and is not intended as investment advice. Permission is granted the reader to generate one copy for

educational use.

This paper is not an exhaustive treatment of the topic. There are no guarantees, and all traders should be prepared

for the unexpected. This author has been known to make errors and reach erroneous conclusions. No doubt, some

of that has crept into this paper.

These notes are a transcript of two workshops given to the Chicago Area MetaStock Group, and are intended for the

use of that group. If any errors are encountered, please contact the group so that corrections can be made:

[email protected] or visit their website at www.elbsoftware.com/metastock/

About the Author

John Jonelis has been an artist, inventor, corporate vice president, and has served on the boards of two companies.

His paintings are represented in various museums and private collections, he holds seven US patents, MBA and

BFA degrees, and has been published in technical journals and websites. He is currently writing a novel while

building and trading systems with his private research group. Other papers by this author can be accessed at the

resource page, http://home.comcast.net/~countertrender/Countertrender.html

Documents

Building a System