Trading Strategies using R - Find Meetup groups near you - Meetup

introductionConnection and data

The questFinal Comments

Trading Strategies using R

The quest for the holy grail

Eran Raviv

Econometric Institute - Erasmus University,http://eranraviv.com

April 02, 2012

Eran Raviv Trading Strategies using R April 02, 2012

http://eranraviv.com



Outline for section 1

1 introduction

2 Connection and data

3 The questSign PredictionFilteringTime Series AnalysisPairs Trading

4 Final Comments




(very) Limited Success

Mean Return 0.000046

Cumulative Return 0.006818323

Proportion of Success 0.472

Total volume 6,039,540

Average Time held 3 hours

# of Trades 531

Average number of trades (day) 3.6

# Trading Days 147

Comission paid for trades 1638

Comission paid for Data 462

SnP return for period 12.00%

The strategy does not even cover transaction costs

Buy and hold is much better for the period measured

Before Comissions

Beating the market is challenging

Report for the period 07/2009 - 07/2010

-0.1

-0.05

0

0.05

0.1

0.15

0.2

1

5

9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

Cumulative Returns

cumsum

Decision was made to reduce

volume





1 introduction



4 Final Comments




For Inter-day, yahoo is fine

nam = c ( 'AON ' , 'MMC' , 'AKS ' , 'BAC ' , . . . ) ; t ck r = so r t (nam)# Most r e c ent 252 days :end<− format ( Sys . Date ( ) , ”%Y−%m−%d” ) # yyyy−mm−dds t a r t<−format ( Sys . Date ( ) − 365 , ”%Y−%m−%d” )l = length ( t ckr )dat = array (dim = c (252 ,6 , l ) )f o r ( i in 1 : l ) {dat0 = ( getSymbols ( t ck r [ i ] , s r c=”yahoo” , from=sta r t , to=end ,auto . a s s i gn = FALSE) ) # Cancel auto . a s s i gn i f you want to

manipulate the ob j e c tdat [ 1 : l ength ( dat0 [ , 2 ] ) , , i ] = dat0 [ , 2 : 6 ]}dat = dat [ 1 : l ength ( na . omit ( dat [ , 1 , 1 ] ) ) , , ]




For Intra-day use IB

IB has extensive API. Connect to their tradingplatform (TWS) using Java and C among others.

Account is not that easy to set up, many forms to fillout and hefty sum to transfer, especially if you wouldlike to day trade.

Jeffrey A. Ryan did outstanding work, we can nowtrade via R.

































Easy:l i b r a r y ( IBrokers )IBrokers v e r s i on 0.9−1: Implementing API Vers ion 9 .64This so f tware comes with NO WARRANTY. Not intended f o r

product ion use ! See ? IBrokers f o r d e t a i l s }con = twsConnect ( c l i e n t I d = 1 , host = ' l o c a l h o s t ' , port

= 7496 , verbose = TRUE, timeout = 5 , f i l ename =NULL)

High frequency data if you have the patience toprogram it.Limitation on the number of requests.In any case not more than one year, but you can storeit.Professional yahoo group at:http://finance.groups.yahoo.com/group/TWSAPI/


http://finance.groups.yahoo.com/group/TWSAPI/







High frequency data if you have the patience toprogram it.

Limitation on the number of requests.In any case not more than one year, but you can storeit.Professional yahoo group at:http://finance.groups.yahoo.com/group/TWSAPI/









High frequency data if you have the patience toprogram it.Limitation on the number of requests.

In any case not more than one year, but you can storeit.Professional yahoo group at:http://finance.groups.yahoo.com/group/TWSAPI/









High frequency data if you have the patience toprogram it.Limitation on the number of requests.In any case not more than one year, but you can storeit.

Professional yahoo group at:http://finance.groups.yahoo.com/group/TWSAPI/









High frequency data if you have the patience toprogram it.Limitation on the number of requests.In any case not more than one year, but you can storeit.Professional yahoo group at:http://finance.groups.yahoo.com/group/TWSAPI/





Sign PredictionFilteringTime Series AnalysisPairs Trading


1 introduction



4 Final Comments





Selected Ideas

Over the years I have backtested many ideas, among others:

Sign Prediction

Filtering

Multivariate time series modelling

Pairs trading

Born to trade, forced to work.





Selected Ideas


Sign Prediction

Filtering


Pairs trading






Selected Ideas


Sign Prediction

Filtering


Pairs trading






Selected Ideas


Sign Prediction

Filtering


Pairs trading






Selected Ideas


Sign Prediction

Filtering


Pairs trading






Selected Ideas


Sign Prediction

Filtering


Pairs trading






Table of Contents

1 introduction



4 Final Comments





Sign Prediction

Sign prediction using:

Logistic Regression (glm)

Support Vector Machine (svm)

♣ library(e1071)

K-Nearest Neighbour (knn)

♣ library(class)

Neural Networks (nnet)

♣ library(nnet)





Sign Prediction - continued

Working with daily returns, so target is to predicttomorrow’s move. (Avoid overnight)

Explanatory variables considered:

I five lags (one week)II Spread between the volume and the rolling average of

most recent 5 days.III Volatility - average of the last five days.






Volatility is measured as the average of three differentintra-day volatility measures which are more efficient(converge faster) than the standard ”sd” estimate:

Parkinson (1980):

σ =√

14Nln2

∑Ni=1(ln

hili

)2

German Klass (1980):

σ =√

1N

∑Ni=1

12(lnhi

li)2 − 1

N

∑Ni=1(2ln2− 1)(ln ci

ci−1)2

Rogers and satchell (1991):

σ =√

1N

∑Ni=1(ln

hili

)(lnhioi

) + (ln lici

)(ln lioi

)







Parkinson (1980):

σ =√

14Nln2

∑Ni=1(ln

hili

)2


σ =√

1N

∑Ni=1

12(lnhi

li)2 − 1

N


ci−1)2


σ =√

1N

∑Ni=1(ln

hili

)(lnhioi

) + (ln lici

)(ln lioi

)







Parkinson (1980):

σ =√

14Nln2

∑Ni=1(ln

hili

)2


σ =√

1N

∑Ni=1

12(lnhi

li)2 − 1

N


ci−1)2


σ =√

1N

∑Ni=1(ln

hili

)(lnhioi

) + (ln lici

)(ln lioi

)






dat0 = ( getSymbols ( t ck r [ 1 ] , s r c=”yahoo” , from=sta r t , to=end ,auto . a s s i gn = FALSE) )

l = length ( dat0 [ , 1 ] )dates0 = ( index ( dat0 ) ) # t r i c k to get t rad ing datest t = NULL ## we now parse i t i n to IB modef o r ( i in 1 : l ) {t t [ i ] = paste ( subs t r ( dates0 [ i ] , 1 , 4 ) , subs t r ( dates0 [ i ] , 6 , 7 ) ,

subs t r ( dates0 [ i ] , 9 , 1 0 ) , sep = ”” )t t [ i ] = paste ( t t [ i ] , ” 23 : 00 : 00 GMT” )}cont=twsEquity ( ' plug your f a v ou r i t e symbol ' , 'SMART ' , 'NYSE ' )mat1 = array (dim = c ( l , 4 00 , 8 ) )#Typical day should have 390

minsf o r ( i in 1 : l ) {m1 = as . matrix ( r eqH i s t o r i c a lData ( con , cont , t t [ i ] , ba rS i z e =

”1 min” ,durat ion = ”1 d” ,useRTH = ”1” ,whatToShow = ”TRADES” , time .

format = ”1” , verbose = TRUE) )mat1 [ i , 1 : l ength (m1 [ , 1 ] ) , ] = m1Sys . s l e e p (14) ## IB r e s t r i c t i o n , WAIT.}






Sample code:l o g i t 1 = glm (y˜ lagy+vo la t+volume , data=dat [ 1 : t1 , ] , f ami ly=

binomial ( l i n k = ” l o g i t ” ) , na . a c t i on=na . pass )summary( l o g i t 1 ) #t1 i s end o f t r a in ing , TT i s f u l l l ength .l i b r a r y ( nnet )nnet1 = nnet ( as . f a c t o r ( y ) ˜ lagy+vo la t+volume , data=dat [ 1 : t1 , ] ,

s i z e =1, t r a c e=T)summary( nnet1 )l i b r a r y ( c l a s s )knn1 = knn ( dat [ 1 : t1 , ] , dat [ ( t1+1) :TT, ] , c l = dat$y [ 1 : t1 ] , k=25,

prob=F)sum(knn1==dat$y [ ( t1+1) ] ) / (TT−t1+1)#Hit r a t i ol i b r a r y ( e1071 )svm1 = svm( dat [ 1 : t1 , 2 : 4 ] , y=dat [ 1 : t1 , 1 ] , type = ”C” )# In sample :sum(svm1$ f i t==dat$y [ ( 1 ) : t1 ] ) / t1# out o f sample :svmpred=pr ed i c t ( svm1 , newdata = dat [ ( t1+1) :TT, 2 : 4 ] )sum( svmpred==dat$y [ ( t1+1) :TT] ) / (TT−t1+1)#Hit r a t i o





Table of Contents

1 introduction



4 Final Comments





Deviation from the mean

Motivation =⇒ Disposition effect, the Voodoo offinancial markets.

Standardise the deviation from the (rolling) mean.





Deviation from the mean

●

●●●

●●●

●●●●●

●●●

●●●

●●●

●

●●●

●●●

●●●●

●●●●

●

●●

●

●●●●

●

●●●●●●

●●●●

●●●●●

●●●●

●

●

●●●

●●●●●

●●●●●●●●●●●●●

●

●●●

●●●

●

●●●

●

●●

●●●

●●

●●●●

●

●

●●●

●●●●

●●

●

●●●●

●

●●

●

●

●●●

●

●

●

●●●●

●

●

●●●

●

●●●

●●

●●

●

●

●

●●●

●

●

●●

●

●

●

●

●●

●

●●●●●●

●●

●●●

●●●

●●

●●●●

●●●●

●

●●●

●

●

●

●●

●

●●●●●

●●●●●

●●●●●●

●●●

●

●●●

●

●●●●●●

●

●●●

●●●

●●●●

●●

●●●●●

●●

●●●

●●●●

●

●●●●

●●●●

●

●●●

0 50 100 150 200 250

500

550

600

650

Google

Days

Price

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●●●●

●●●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●

●●●●●●●

●●●

●●●●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●

●●●●●●

●●●●●●

Histogram for Z

Freq

uenc

y

−4 −2 0 2

05

1015

20

●●

●

●

●●

●●

●

●

●●

●●

●

●

●

●

●

●●●

●

●●

●

●

●

●●●

●●●●●●●●●●

●

●●

●

●

●●●

●

●●

●●

●

●

●

●

●●●●

●

●●

●

●●

●

●

●●●●

●

●●

●

●

●●●

●

●●●

●

●

●●

●●●●●

●●●●

●●

●●

●

●

●

●●●

●●

●●

●

●●●●

●●●●●●

●●

●

●●●

●●●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●●●●

●●●●●

●●

●●

●●

●

●

●●●

●

●

●

●●

●●

●

●

●

●●●●●

●

●●●

●●●●●●●●

●●

●●

●

●

●

●

●●●●

●●●

●

●

●●●●

●●●●●

●

●●●●●

●●●●●

●

●●●●

●●

●●

●

●●●

●

●

●

●

0 50 100 150 200 250

−4−2

02

Days

Z





Table of Contents

1 introduction



4 Final Comments





Motivation

Momentum in Microstructure - Dermot Murphy andRamabhadran S. Thirumalai (Job Market Paper -2011)

Are You Trading Predictably? -Steven L. Heston,Robert A. Korajczyk ,Ronnie Sadka, Lewis D.Thorson. (2010)

We find predictable patterns in stock returns. Stockswhose relative returns are high in a given half-hourinterval today exhibit similar outperformance in thesame half-hour period on subsequent days. The effectis stronger at the beginning and end of the tradingday. These results suggest...





Motivation

Momentum in Microstructure - Dermot Murphy andRamabhadran S. Thirumalai (Job Market Paper -2011)

Are You Trading Predictably? -Steven L. Heston,Robert A. Korajczyk ,Ronnie Sadka, Lewis D.Thorson. (2010)

We find predictable patterns in stock returns. Stockswhose relative returns are high in a given half-hourinterval today exhibit similar outperformance in thesame half-hour period on subsequent days. The effectis stronger at the beginning and end of the tradingday. These results suggest...





VAR models

For each day t = {1, ..., T}, the return of half an hourk = {1, ..., 13} , and the lag number p = {1, ..., P}:y1,ty2,t

...yk,t

=

c1c2...ck

+

a11,1 a11,2 · · · a11,ka12,1 a12,2 · · · a12,k

......

. . ....

a1k,1 a1k,2 · · · a1k,k

y1,t−1y2,t−1

...yk,t−1

+ · · ·+

ap1,1 ap1,2 · · · ap1,kap2,1 ap2,2 · · · ap2,k

......

. . ....

apk,1 apk,2 · · · apk,k

y1,t−py2,t−p

...yk,t−p

+

e1,te2,t

...ek,t

Problem: for P = 1, how many parameters?





VAR models

For each day t = {1, ..., T}, the return of half an hourk = {1, ..., 13} , and the lag number p = {1, ..., P}:y1,ty2,t

...yk,t

=

c1c2...ck

+

a11,1 a11,2 · · · a11,ka12,1 a12,2 · · · a12,k

......

. . ....

a1k,1 a1k,2 · · · a1k,k

y1,t−1y2,t−1

...yk,t−1

+ · · ·+

ap1,1 ap1,2 · · · ap1,kap2,1 ap2,2 · · · ap2,k

......

. . ....

apk,1 apk,2 · · · apk,k

y1,t−py2,t−p

...yk,t−p

+

e1,te2,t

...ek,t

Problem: for P = 1, how many parameters?





VAR models (cont’d)

Possible solution =⇒ Dimension Reduction.

Stepwise Regression, Lasso, Variable selection(according to some Information Criteria), PrincipalComponent Regression, Ridge Regression, BayesianVAR and many more.

Very nice vars package to start you off, though as mostbuilt-ins, not flexible enough. (e.g. rolling windowsand/or shrinking)





VAR models (cont’d)

Possible solution =⇒ Dimension Reduction.

Stepwise Regression, Lasso, Variable selection(according to some Information Criteria), PrincipalComponent Regression, Ridge Regression, BayesianVAR and many more.

Very nice vars package to start you off, though as mostbuilt-ins, not flexible enough. (e.g. rolling windowsand/or shrinking)





Table of Contents

1 introduction



4 Final Comments





Pairs Trading

Well known and widely used. (e.g. Statistical Arbitragein the U.S. Equities Market, Marco Avellaneda andJeong-Hyun Lee (2008))

Suitable for the conservative mind. (we see why in aminute..)





Pairs Trading







Pairs Trading







Pairs Trading (cont’d)

The Idea:

ra = βarm + ea

rb = βbrm + eb

rab = wa(βarm + ea) + wb(βbrm + ea)

= rm(waβa + wbβb) + noise

and so with weights wa = − βbβa−βb

and wb = 1− wa we can

net out the market. (and other factors if you will)






Choose symbols with similar properties.

Net out the market and create the spread:

## sp1 = stock p r i c e 1 , g=s i z e o f moving window ,## n = length ( sp1 )

f o r ( i in g : n ) {bet0 [ i ]=lm( sp1 [ ( i−g+1) : ( i −1) ] ˜ sp2 [ ( i−g+1) : ( i −1) ] ) $ co e f [ 1 ] #

# note −> i−1bet1 [ i ]=lm( sp1 [ ( i−g+1) : ( i −1) ] ˜ sp2 [ ( i−g+1) : ( i −1) ] ) $ co e f [ 2 ]

spread [ , i ]=sp1 [ ( i−g+1) : i ]− rep ( bet0 [ i ] , g )−bet1 [ i ] *sp2 [ ( i−g+1) : i ]}

Text book example (actually from: QuantitativeTrading: How to Build Your Own Algorithmic TradingBusiness )

The GLD and GDX spread






Choose symbols with similar properties.

Net out the market and create the spread:

## sp1 = stock p r i c e 1 , g=s i z e o f moving window ,## n = length ( sp1 )

f o r ( i in g : n ) {bet0 [ i ]=lm( sp1 [ ( i−g+1) : ( i −1) ] ˜ sp2 [ ( i−g+1) : ( i −1) ] ) $ co e f [ 1 ] #

# note −> i−1bet1 [ i ]=lm( sp1 [ ( i−g+1) : ( i −1) ] ˜ sp2 [ ( i−g+1) : ( i −1) ] ) $ co e f [ 2 ]

spread [ , i ]=sp1 [ ( i−g+1) : i ]− rep ( bet0 [ i ] , g )−bet1 [ i ] *sp2 [ ( i−g+1) : i ]}

Text book example (actually from: QuantitativeTrading: How to Build Your Own Algorithmic TradingBusiness )

The GLD and GDX spread






The GLD and GDX spread:





Pairs Trading Issues

Estimation of the market neutral portfolio is tricky:

Price levels or price changes?

Stability over time

Errors on both sides. (both y and x are measured witherrors)








Stability over time









Stability over time






Pairs trading issues

Stability over time:





Pairs trading issues

Errors on both sides:

sta = αstb + ea

stb = βsta + eb

α̂ 6= 1

β̂⇓

Portfolio is different and will depend on which instrumentgoes on the LHS and which on the RHS.





Pairs trading - possible solutions


flip a coin (solid option)

average the estimates





















Pairs trading - possible solutions (cont’d)

Stability over time

Choose window length that fits your style, the shorterthe more you trade.

Recent paper (though in different context) suggests toaverage estimates across different windows to partiallyhedge out uncertainty. (M. Hashem Pesaran, Andreas Pick.Journal of Business and Economic Statistics. April 1, 2011)

Kalman filter the coefficients.






Stability over time

Choose window length that fits your style, the shorterthe more you trade.

Recent paper (though in different context) suggests toaverage estimates across different windows to partiallyhedge out uncertainty. (M. Hashem Pesaran, Andreas Pick.Journal of Business and Economic Statistics. April 1, 2011)

Kalman filter the coefficients.






Errors on both sides, two highly correlated possiblesolutions:

Demming regression (1943). (Total least squares - justminimize numerically both sides simultaneously)

Geometric Mean Regression - force coherence through:

sta = αstb + ea

stb = βsta + eb

γ̂ =

√α̂× 1

β̂





1 introduction



4 Final Comments




Miscellaneous remarks

Trading costs!, consider it when backtesting.

You cannot be too careful, stay pessimistic.

Adopt rigorous robustness checks, differentinstruments, different time frames and even differentmarkets.

Use paper money for at least a full quarter, it will helpyou handle operational problems. (e.g. outages andtime zones issues)

It is (very) stressing work, know it before you start.

Know what you are doing, what is your edge? why it is(not) there?






















































THANKSand good luck at the tables..


Documents

Trading Strategies using R - Find Meetup groups near you - Meetup