Quantified News Based Trading - Princeton UChicago - Rajib Ranjan Borah

8/10/2019 Quantified News Based Trading - Princeton UChicago - Rajib Ranjan Borah

1/176

Quantified News based Trading:

is it the next big thing in algorithmic

trading ?

Rajib Ranjan Borah

Nov 8, 2013PrincetonUChicago Quant Trading Conference


2/176

Agenda

Background - how is news quantified

Profitability using quantitative news analysis

Machine learning techniques for designing quant news strategies

Q&A


3/176

Agenda




Q&A

How is news quantified Profitability Machine learning techniques QA


4/176

.

The world runs on information and few areas as directly so as in

finance



5/176

Historical Perspective

1. Rothschild:



6/176


1. Rothschild:

A family network spread across Europe (Frankfurt, London,

Paris, Naples, Vienna)



7/176


1. Rothschild:


Paris, Naples, Vienna) enabled obtaining financial

information before peers



8/176


1. Rothschild:




Knowledge of Battle of Waterloo result one full day before

others



9/176


1. Rothschild:





others largest private fortune in the world



10/176


1. Rothschild:






2. Reuters:



11/176


1. Rothschild:






2. Reuters:

News service used pigeons & telegraph in 1850s to becomefastest news disseminator


H i ifi d P fi bili M hi l i h i


12/176


1. Rothschild:






2. Reuters:


Continued focus on being the fastest news source


H i tifi d P fit bilit M hi l i t h i


13/176


1. Rothschild:






2. Reuters:


Continued focus on being the fastest news source $12.4

billion conglomerate


H i tifi d P fit bilit M hi l i t h i QA


14/176


How have things progressed since 1850s ?



15/176



16/176



1850s : Carrier pigeons

1860s : Telegraph




17/176




1860s : Telegraph

1960s : Teleprinter




18/176




1860s : Telegraph

1960s : Teleprinter 1980s : Electronic network (and internet)




19/176




1860s : Telegraph

1960s : Teleprinter 1980s : Electronic network (and internet)

2000s : Machine Readable News a.k.a. Quantitative News




20/176

What is Quantitative News Trading?

News is the first order factor that affects prices, volume,

volatility of stocks, currencies, commodities, etc




21/176




Computer programs that scan news articles & quantify them




22/176





-> can respond to price moving factors faster than humans




23/176






-> can monitor a vaster amount of news reports than humans




24/176







This field is known as Quantitative News Trading

q y g q Q


25/176



26/176








Apart from trading, quantification of news is also utilized in

Media evaluation

Market research

Brand & reputation management

Political analysis

q y g q



27/176


Sample output of a News Analytics feed: News

represented by numbers



28/176








Apart from trading, quantification of news is also utilized in

Media evaluation

Market research

Brand & reputation management

Political analysis



29/176








During the 200 milliseconds a human is reading the latest news headline, a

trading program will have downloaded the entire article, analyzed its

meaning, & traded based on the content



30/176













31/176











How do you quantify news reports and articles ?



32/176

Quantifying News - 1. Sentiment

News articles are assigned a score called sentiment



33/176



Sentiment says whether the article has a positive / negative or

neutral tone



34/176




neutral tone

(Sale of Apple iPhones drop = -ve sentiment)



35/176




neutral tone


Sentiment at document level is different from sentiment at

entity level



36/176




neutral tone


Sentiment at document level is different from sentiment at

entity level(Samsung beats Apple in smart phone sales = -ve sentiment for

entity named Apple, +ve sentiment for Samsung)



37/176


How is sentiment scored ?



38/176



Naive parser: based on word count ofve / +ve keywords



39/176




Company Xs sales were good

Company Xs sales were fantastic

(Both have one positive keyword, but the intensity differs)



40/176




Discriminated parser: weighted word count The results were good, not bad.

The results were bad, not good.

(Both score equally. Both have the same words - but mean completely

opposite)



41/176




Discriminated parser: weighted word count Grammatical parser: which verbs work on which objects.

check linguistic semantics



42/176






Machine Learning: From the data and the answers, try to find

the factors



43/176






Machine Learning: From the data and the answers, try to find

the factors Generate bag-of-words: distance of subject from these sentiment

words

Overfitting (and large vector sets), hitch-hiking and ignorance of

linguistic structure



44/176


Scoring sentiments: grammatical parsing



45/176



A database of words & phrases against which the article is

searched



46/176



A database of words & phrases against which the article is

searched

Which verbs act on which objects


47/176



48/176


Scoring sentiments: grammatical parsing issues



49/176



Linguistic structures like negation, double negation, sarcasm,

intensification, hanging lemma


50/176



51/176





(negation: Company X did not become the best in the world

double negation: Company X did not do bad

sarcasm: With such an attitude, X is sure to become the best firm

intensification: Company X did terribly well

hanging lemma: Company X loses lawsuit against company Y. They will

have to pay $1billion USD )

Word Sense Disambiguation - same word, different meanings



52/176





(negation: Company X did not become the best in the world

double negation: Company X did not do bad

sarcasm: With such an attitude, X is sure to become the best firm

intensification: Company X did terribly well

hanging lemma: Company X loses lawsuit against company Y. They will

have to pay $1billion USD )

Word Sense Disambiguation - same word, different meanings Company X received a fine

X is doing fine X sells fine grained sand, etc



53/176

Quantifying News - 2. Relevance

Is Sentiment good enough to quantify a news report?



54/176



A news article might:

be predominantly about a company



55/176





mention that company and others as well



56/176






mention that company in passing in the article



57/176






mention that company in passing in the article

Relevance measures how relevant a news article is for aparticular company



58/176


How is relevance scored ?


59/176


60/176


61/176



62/176



How many companies are mentioned in the news article

Is the company mentioned in the headline as the

subject/object

(Headline:UBS downgrades HSBC is not relevant to UBS)

In which sentence number is the company first mentioned

Length of the article & how many times is the firm mentioned



63/176



How many companies are mentioned in the news article

Is the company mentioned in the headline as the

subject/object

(Headline:UBS downgrades HSBC is not relevant to UBS)

In which sentence number is the company first mentioned

Length of the article & how many times is the firm mentioned Number of sentiment words & total words in article


64/176


65/176



66/176


Issues with calculating relevance

Requires synonym database:



67/176




IBM

International Business Machines

I.B.M.

Big Blue



68/176




IBM

International Business Machines

I.B.M.

Big Blue

BAML Bank of America

Merrill Lynch

Bank of America Merrill Lynch

Merrill

BoA


69/176



70/176

Quantifying News - 3. Novelty

Often the news article is not reported in its entirety, but inmultiple spurts



71/176



Alert

News Article

Update Append



72/176



Alert

News Article

Update

Append

Moreover, multiple news

sources report same news



73/176



Alert

News Article

Update

Append

Moreover, multiple news

sources report same news

News also cause price

changes which themselves

become news



74/176


If we do not keep track & respond to repeated instances ofthe same news



75/176


If we do not keep track & respond to repeated instances ofthe same news => we will end up repeating our actions

manifold for the same event



76/176


If we do not keep track & respond to repeated instances ofthe same news => we will end up repeating our actions

manifold for the same event

Therefore every news article should be checked for newnessor novelty before responding



77/176


How is novelty measured ?



78/176



The keywords in the current news article are compared to

historical articles about that company for similarity of digital

fingerprints



79/176





fingerprints

A linked articles count is generated



80/176





fingerprints


Novelty is reported for

Within same news feed novelty (i.e. all Bloomberg news articles only)



81/176





fingerprints


Novelty is reported for

Within same news feed novelty (i.e. all Bloomberg news articles only)

Across all news feeds novelty (i.e. across Reuters, Dow Jones,

Bloomberg articles)



82/176

Quantifying News - 4. Market Impact

Different types of news articles have different impacts on theprice of the asset



83/176

Quantifying News - 4. Market Impact

Different types of news articles have different impacts on theprice of the asset

Another aspect of relevance is the likely market impact of the

news article


84/176

f



85/176

Quantifying News - News Types

Types of news:

Accounting news

Earnings

Trading updates (broker action, market commentary)

Guidance

Financial issues (buybacks, dividends, equity offerings, etc)

Regulatory filings


86/176

Q if i N N T



87/176

Quantifying News - News Types

Types of news based on time of news report

Asynchronous / unexpected

Synchronous / fixed releases

Q if i N K F



88/176

Quantifying News - Key Factors

While the following are the four key inputs:


89/176

Q tif i N K F t



90/176



Sentiment

Relevance

Novelty

Market Impact

Some news analytics based strategies use other factors as well

Q tif i N 5 V l



91/176

Quantifying News - 5. Volume

The number of news articles on the same topic can be a usefulinput to validate the impact

Q tif i N 5 V l



92/176



Volume of news in Social Media also checked sometimes

Q tif i N 5 V l



93/176



Volume of news in Social Media also checked sometimes

News Analytics strategies also check market based qualitative

parameters along with news -> these help check if reaction to

news is not already factored in

Trading Volume in last 24 hours (and historical average volume)

Price change in last 24 hours


94/176


95/176

Quantifying News 6 Social Media



96/176

Quantifying News - 6. Social Media

Long term trading strategies try to gauge market sentiment fromthe plethora of information in the social media front

Search engine volume counts (e.g. Google Trends) - global

search for news keywords.

Can be used to confirm market impact of news




97/176




search for news keywords.Can be used to confirm market impact of news

Facebook, Twitter - user sentiment evaluated at macro level.




98/176




search for news keywords.Can be used to confirm market impact of news

Facebook, Twitter - user sentiment evaluated at macro level.

Many tools use certified twitter/facebook feeds only

Quantifying News Key Factors



99/176



Sentiment

Relevance

Novelty

Market Impact

Some news analytics based strategies use other factors as well Volume

Social Media

Quantifying News Market Psyche



100/176

Quantifying NewsMarket Psyche

News Analytics tools calculate Market Psychology Indices -evaluating broad psychological sentiments from global news




101/176



Country : sentiment, conflict, fear, joy, optimism, trust,

uncertainty, urgency, violence, government corruption,government instability, social unrest, default, inflation, credit

tightening, etc




102/176





tightening, etc

Equity: Gloom, Anger, Innovation, Stress, Optimism, Earnings

Expectations, Market Risk, Market Forecast




103/176





tightening, etc


Expectations, Market Risk, Market Forecast Currency: Forecast, Currency Peg Instability, Carry Trade


How is news quantified

Profitability

Machine learning techniques QA


104/176





tightening, etc


Expectations, Market Risk, Market Forecast Currency: Forecast, Currency Peg Instability, Carry Trade

Agriculture: Acreage cultivated, weather damage, subsidies,

production volume, supply vs demand, surplus vs shortage,

price up


105/176




106/176


Agenda



107/176

Agenda




Q&A

Is it profitable ?



108/176

Is it profitable ?

Are computers smart enough to read news and make profitabletrades?


109/176


110/176

Where Quantified news work



111/176


Machines are faster at responding to events than humans




112/176


Machines are faster at responding to events than humans

Machines can process a much vaster amount of information

without any fatigue




113/176


Machines are faster at responding to events than humansLow latency event based trading (first to respond)


without any fatigue




114/176




without any fatigue

Analyze broad spectrum of news to formulate broad views




115/176




without any fatigue





116/176






117/176

Q





118/176

Q





119/176

Q

Low latency event based trading (first to respond)




120/176

Q


For synchronous (fixed releases) expected events (earnings

releases/ economic figures)

Company figures provided in xml format instead of text




121/176





Economic figures provided in binary format instead of textual

news articles




122/176





Economic figures provided in binary format instead of textual

news articles

For asynchronous / unexpected news

Are quantification algorithms robust enough to calculate

trust-worthy sentiment, relevance, novelty scores ?

Opportunities : initial under-reaction



123/176

pp

Quantified news driven trades work even when the trade is doneat the end of the day

(under-reaction to news immediately. Tetlock, et al)

Lateendofdayresponsealsoprofitable



124/176

y p p

Trading the news immediately = very profitableAt a broad level there is underreaction to news => entering into

trades at the end of the day also makes profits

Long short strategy returns



125/176

Filtering sentiments increase profits



126/176

Increasing threshold from 90 to

95 percentile increases returns

from 55 to 138 bps in 3 days

Certain sectors more profitable



127/176

Moving from Non-Cyclicals to

Financials increased the profit

from 135BP to 147BP


128/176

Small cap firms more profitable



129/176

Smaller Cap firms show greater response to extreme sentimentnews event

(bigger firms have greater scrutiny)

Filter & trade fewer stocks



130/176

More is not better. Quality over quantity

Trading only stocks with very high sentiment/relevance is

better

Hedged (market-neutral) is better



131/176

Long +ve sentiment stocks onlyOR

Short -ve sentiment stocks only. Will fail in different regimes

Being long +ve sentiment stocks & short -ve sentiment stocks

at the same time gives consistent returns

Volatility regimes and news



132/176

Volatile vs stable Economic regimes

In more volatile markets people tend to react less strongly to

positive news and react more strongly to negative news

Surprises are more profitable



133/176

Bigger moves happen when there is news in

Stocks with low beta (i.e. surprises happen to sleepy stocks)




134/176






135/176



VIX is low (i.e. surprises during calm times)


136/176




137/176




When markets are improving (i.e. surprise to mostly longposition holders)




138/176








139/176





Strategy variation - sentiment changes



140/176

Instead of absolute sentiment scores, look at changes insentiment scores of firms

Bought stocks with highest increase in sentiment

Shorted stocks with highest decrease in sentiment

Strategy variation - bottom fishing



141/176

Bottom - fishing / turnaround stories Buying stocks with reversal in sentiment from grossly

negative (a lot of the stocks turned out to be buybacks)

Strategy variation - trading volatility



142/176

News articles definitely lead to increased volatility, eventhough direction of move might be difficult to predict

through news analytics

Take vega positions (var-swaps) using options in anticipationof increased volatility

Generating Alpha



143/176

Soft (opinion based) vs. Hard (fact based) newsHard news has a stronger short term reaction than soft news

Source: RavenPack, FactSet, Macquarie Research, September 2012


Generating Alpha


144/176

Scheduled/expected vs. Unscheduled/unexpectedInvestors react more strongly to unscheduled/ unexpected

news than scheduled/ expected



Generating Alpha


145/176

Forecast vs Actual earningsInvestors react more strongly to forecasts than actual earnings

news



Generating Alpha


146/176

Guidance vs Actual earningsInvestors react more strongly to guidance to actual earnings



Generating Alpha


147/176

News type Event Study Results


To summarize



148/176

News Analytics works best with

Small cap stocks

Sectors like pharma, banking, etc

Stocks with low beta

When VIX is low

When markets are improving

Hard news (vis-a-vis Soft news)

Unscheduled news events (vis-a-vis scheduled news events)

Being market-neutral

Doing fewer stocks, but those with stronger signals

Quantifying News - Where it fails ?



149/176

On Sep. 7, 2008 Googles newsbots picked up an old 2002story about United Airlines possibly filing for bankruptcy

Quantifying News - Where it fails ?



150/176

On Sep. 7, 2008 Googles newsbots picked up an old 2002story about United Airlines possibly filing for bankruptcy

UAL stock dived immediately

Quantifying News - Where it fails?



151/176

News analytics were taught that Osama-Bin-Laden, andkilled had -ve sentiments for the markets

Quantifying News - Where it fails?



152/176

News analytics were taught that Osama-Bin-Laden, andkilled had -ve sentiments for the markets

On May 2 2012 when news reporting Osama Bin-Landen

killed were published, news bots treated this as a negativenews article and sold stocks


153/176

Quantifying Newschallenges



154/176

Languages like Chinese and Japanese with large number ofalphabetic symbols and complex grammar


155/176


156/176

Agenda



157/176




Q&A

Machine Learning methodologies



158/176

Traditional approach => formulate hypothesis based onexperience/expertise, validate statistically using historical data




159/176


Machine learning approach =>




160/176


Machine learning approach => output + raw data fed into a

system. System reports factors within data that lead to output




161/176


Machine learning approach => output + raw data fed into a

system. System reports factors within data that lead to output

Three broad approaches

Tree

Forest

Planet

Machine Learning - TREE method



162/176

Output: Post-event abnormal resultsInput: Quantitative news analytics




163/176





164/176


Issues: Overfitting

(works with training data

does not work on real data)




165/176


Issues: Overfitting



Solution: Pruning




166/176


Issues: Overfitting



Solution: Pruning

Machine Learning - FOREST method



167/176

Multiple factors might impact output




168/176


Instead of one tree to solve everything,

have a forest of trees




169/176







170/176




Each tree has a vote in the output.

Weightage of vote depends on accuracy

of that tree


171/176

Machine Learning - PLANET method



172/176

Instead of linear relationships between input and output,

Planet breaks the variable space into sections, fits linear

functions within those sections


173/176


174/176

Text Mining: An example



175/176

Converting a lineof news into

metadata to be

used for analysis

or trade


176/176

Documents

Quantified News Based Trading - Princeton UChicago - Rajib Ranjan Borah