Can tweets help predict a stock's price movements?

26 | The Journal of the CFA Society of the UK | www.cfauk.org

Feature | Professional Investor

Web users’ interactions and commentaries in the social media space can reflect current opinions, views and experiences, and therefore contain helpful information for market research. Consumers who profess stronger positive affinity with a certain brand are likely to have a higher customer lifetime value, a predictor of the net present value of profits from a customer over the entire future relationship with him/her. The evidence so far is too little to demonstrate consistent results. However, this new avenue demands further investigation with advanced statistical analysis and larger scale application.

INTRODUCTIONIn the early days when internet search algorithms were being developed, who could have imagined that search data could be used to predict the future? Yet here we are in 2012 witnessing it. Many organisations have found that data extracted from specific searches can predict – or at least model the future.

The Bank of England (BoE) is just one organisation that is convinced that appropriately interpreted search data can act as an indicator of future economic trends. In June 2011, a team of researchers from the BoE released a report illustrating how

Fernan Flores asks whether the analysis of tweets or other social media postings could be a useful predictor of market movements, as it has been demonstrated in the case of Google search data.

Tweet: “@cfauk – is it true that tweets can help predict a stock price movement?”

The Journal of the CFA Society of the UK | www.cfauk.org | 27

Professional Investor | Feature

results extracted from Google search data could predict changes in unemployment and even house prices.

Being not a fan of social media sites, I had never used Twitter, a micro blogging site, until I read an article describing it as the new Google. As a market research analyst and consequently a fan of Google, I was intrigued and registered for Twitter to see what the buzz was about.

Twitter has indeed a search function that allows anyone to browse through tweets, postings or status updates, sometimes in real time. In fact, the research results by seeking out key words from tweets proved to be very useful when I undertook some competitive intelligence work for a client to check about its competitor’s customer service. This was quite a revelation.

Web users’ interactions and commentaries in the social media space can reflect current opinions, views and experiences and therefore contain helpful information for market research. Could the analysis of tweets or other social media postings be a useful predictor of market movements though, as it has been demonstrated in the case of Google search data?

Derwent Capital, a company which was originally established as a hedge fund that used consumer tweets in its trading strategy but has now repositioned itself as a technology provider giving traders and investor access to its proprietary platform, said in an article published in August 2011 that based on its research and testing of randomly selected unstructured data from Twitter that its algorithm, which helps classify a tweet into a sentiment (e.g. alert, vital, happy), helped predict movements in liquid stocks.

A similar strategy was replicated by the University of Manchester and Indiana University in a research paper (Bollen, Mao, and Zeng, 2010), showing that Twitter data analysed for sentiment predicted around 87.6% of the movements in the Dow Jones industrial average. The study was based on an assumption used in behavioural finance, which states that “financial decisions are significantly driven by emotion and mood… therefore, [it is] reasonable to assume that the public mood and sentiment can drive stock market values as much as news.”

ANALYSISIn order to explain unstructured tweets, many social media monitoring and analytics (SMMA) firms like Derwent Capital have developed algorithms that categorise tweets (or any social media postings) as positive, neutral or negative. The tweets are further classified so that words that express stronger emotions are classified at the extreme ends of a Likert scale such as the ones illustrated in Chart 1 above.

The hypothesis that social media can be a strong indicator of financial performance is based on the principle that consumers who profess stronger positive affinity with a certain brand will have a higher customer lifetime value, a predictor of the net present value of profits from a customer over the entire future relationship with him/her. If a brand or organisation has more customers with stronger positive (or less negative) affinity, it should have a positive financial outlook, which is reflected through a strong stock performance.

To prove this relationship at a basic level, I plotted the proportion of positive and negative sentiments against the closing stock price of Apple (see Figures 1 and 2) and Microsoft (see Figures 3 and 4). Because of the volatility of the data, especially the sentiments, I used the data’s three-day moving average standardised with z-scores in order to compare the movements in the stock price and the sentiments more evenly.

Chart 1: Likert Scale

“Financial decisions are significantly driven by emotion and mood…therefore, it is reasonable to assume that the public mood and sentiment can drive stock market values as much as news.”

Apple annoys me! I will never buy an

iPhone again.

My iPhone is getting

problematic.

My iPhone is working ok.

I enjoy using my iPhone.

I love my new iPhone! I strongly recommend that

everyone buys one too!

1 2 3 4 5

Positive sentimentsNegative sentiments

28 | The Journal of the CFA Society of the UK | www.cfauk.org

Apple (Jan - Dec 2011)


Microsoft (Jan - Dec* 2011)


Apple (Dec 2011 - Jan 2012)

Feature | Professional Investor

As can be seen in Figures 1 and 2, the correlation coefficient between the stock price and sentiments is very weak for Apple and actually counter-intuitive as the positive sentiments trend is negatively correlated with the stock price.

For Microsoft, a relationship seems to exist especially for positive sentiments. As highlighted in Figures 3 and 4, there are days when either the positive or negative sentiments clearly moved along with the changes in stock price (as highlighted by the blue vertical lines).

While the accuracy of the technology developed by SMMA firms in data mining has considerably improved over the years, removing spam or filtering only relevant information remains a challenge with the best technology achieving only an accuracy level of between 75%-85% and the majority achieving an accuracy level of between 50%-60%.

A deep-dive analysis of Apple verbatims reveals that a considerable number of statements analysed refers to either apple, the fruit, or apple juice. It is therefore not surprising that the relationship between the sentiments and Apple’s stock price hardly exists at all.

In contrast, the data mining technology has more accurately analysed Microsoft given the uniqueness of the brand as a term. The resulting correlation for Microsoft over a year, however, remains weak. It could be possible that while verbatims for Apple include irrelevant information, analysis for Microsoft may have excluded tweets that refer to Microsoft but have been omitted because consumers may have used their own jargon when spelling the brand or have unintentionally misspelled it (e.g. MS, Macrosoft, Mikrosoft, Microsof ).

Figures 3 and 4:Microsoft full year 2011 (positive and negative sentiments)






Source: Yahoo! Finance and Twitter

Figures 3 and 4. Z-scores of Microsoft’s closing stock price in NASDAQ versus z-scores of positive sentiments (Figure 3) and negative sentiments (Figure 4) from Twitter. (Note that z-scores of negative sentiments are shown in reverse order as a decrease in negative sentiment is expected to have a positive impact while an increase in negative sentiment is expected to have a negative impact on a stock’s performance.)

Figures 1 and 2:Apple full year 2011 (positive and negative sentiments)







Figures 1 and 2. Z-scores of Apple’s closing stock price in NASDAQ versus z-scores of positive sentiments (Figure 1) and negative sentiments (Figure 2) from Twitter. (Note that z-scores of negative sentiments are shown in reverse order as a decrease in negative sentiment is expected to have a positive impact while an increase in negative sentiment is expected to have a negative impact on a stock’s performance.)






r = -0.26

r = 0.39

r = 0.14

r = -0.26

I conducted regression analysis and made various combinations of analysis accounting for potential lag, comparing the weighted average score of all sentiments (i.e. rating extremely positive statements a 5, a relatively positive statement a 4, a neutral statement a 3, a relatively negative statement a 2 and an extremely negative statement a 1) and comparing the net sentiment (i.e. the resulting proportion of sentiments when negative is deducted from positive) but none of the resulting analysis proved that the sentiments have a strong relationship with a brand’s stock price.

With some effort, I manually cleaned hundreds of Apple tweets (i.e. removing tweets that refer to apple, the fruit, or apple juice) from December 2011 until January 2012. The resulting comparison as shown in Figure 5 illustrates that tweets that are more accurately filtered can potentially be more effective in predicting a brand’s stock price, achieving a correlation coefficient of 0.85.

CONCLUSIONWhile manually cleaned Twitter sentiments, at least for Apple in this example, shows that consumer sentiment movements movements can have a strong correlation to a company’s stock price movements, the evidence so far is too little to demonstrate consistent results. Clearly, this new avenue consisting of exploitating Twitter or other social media websites demands further investigation with advanced statistical analysis and application on a larger scale to ascertain the relationship between the two data sets.

With the rapid progress of technology in this field, especially with search algorithms becoming more and more clever, it is likely that the capability to demonstrate a correlation will improve across time.

Can this work for non-consumer brands (e.g. BHP Billiton)? Can sentiments on brands really have an impact on the stock

price of the company that owns them (e.g. PG tips, Bovril and Persil owned by Unilever)? Can tweets from non-English speaking countries and consumers, which are continuously increasing in share as a proportion of total global tweets, weaken or strengthen the relationship between sentiments and stock price? These are just a few of the questions that we have not even begun to address. Yet as technology develops, this will spread into other compatible areas, geographies and cultures.

Given these issues, using tweets or any social media data for trading strategy needs further exploration to strengthen the case for it. But perhaps, based on Everett Rogers’ theory of “Diffusion of Innovation” this may not be necessary for innovators and early adopters – the consumer segments which adopt technology ahead of the rest of the population. Given the speed of technological innovation in data mining, combined with advanced statistical analysis, I am confident that using social media as a highly reliable predictor of stock price movements can be achieved much sooner than expected. However, when this point happens and when everyone else starts to use insights from tweet sentiments for trading, then the opportunity for arbitrage will have disappeared. ■

The Journal of the CFA Society of the UK | www.cfauk.org | 29

Professional Investor | Feature

Profile

Fernan Flores

Fernan Flores is a freelance market research analyst and director at Zapienza, a Canary Wharf-based market research consulting firm that specialises in the technology and finance sectors, which he established after completing his MBA degree from the Cambridge Judge Business School. Apart from the technology and finance sectors, he also does a considerable amount of work in the not-for-profit sector and specialises in the deployment of technology to solve healthcare issues in developing markets. He has passed the Level I exam of the CFA Program and is a member of the CFA UK marketing and communications committee.


Figure 5:Apple 2 months December 2011 - January 2012







Figure 5. Z-scores of Apple’s closing stock price in NASDAQ versus z-scores of positive sentiments using data that are further filtered manually.

r = -0.85

Economy & Finance

Can tweets help predict a stock's price movements?