Real-Time News Analytics With Semantic Big Data Technologies

Real-Time News AnalyticsWith Semantic Big Data Technologies

Dr. Volker Stümpflen

and

Michael Schramm

Clueda AG

1.4.2014

Clueda AG

Clueda

Founded 2012

Spin-Off Institute for Bioinformatics and Systemsbiology of the Helmholtz Zentrum München

Real-time software solutions for semantic and associative knowledge processing and analysis

>40 man years R&D

30 employees

Partner: Baader Bank AG

Winner Best in Big Data Award 2013

2

Clueda AG

Clueda AG

Why Big Data

Storage is cheap

Data is globally accesible

4

Clueda AG

Big Data Processing is Possible (for Everyone)

5

Clueda AG

Newsflood

Millions of financial instruments

X traders and analysts

500.000 news p.d.~4 bn sentences p.a.

From stocks toderivatives

Increasing

Decreasing time forincreasing information

Is constant and small

From news agenciesto social mediachannels (Blogs, Tweets)

Strongly increasing

6

Clueda AG

News Moves Markets

7

Time

Pric

e

News published

Clueda analysis readytrader is buying

News reading

Automated analysis Commercialadvantage

News reading finishedtrader is buying

Clueda AG

Big Data Problem: Big Data – Big Noise

Junk-In -> Junk-Out

8

Clueda AG

Big Data Problem: Correlation vs. Causality

9

Clueda AG

Needle in a Haystack

10

Clueda AG

User-Centric Decision Making

11

SeeConcepts, relations and events as they happenin multiple information sources

UnderstandTrends, mood and relationships using semantics and systems biology approaches

AnswerQuestions that only specialists could answer before

Data

Information

Knowledge

Real-timeengine

Clueda AG

Market Moving Influences

12

InsiderKnowl.

Market Moving

Events

Mood

InformationSentiment

Clueda AG

Elementary Processing Steps

13

Recognizing Concepts(Companies, Persons, ...)

Advanced Analytics(e.g. Sentiment)

Generating Knowledge Networks

Recognizing Relations and Events

Clueda AG

Simple Detection And Utilizing Of Concepts

Applications and Problems

14

Source : Preis, T., Moat, H. S. & Stanley, H. E. Quantifying Trading Behavior in Financial Markets Using Google Trends. Sci. Rep. 3, 1684 (2013).

Clueda AG

Concept Detection

Recognizing the meaning of unknown words

Self-learning capabilities based on machine learning approaches

After initial training knowledge base ist extended automatically

15

Clueda AG

Real-Time Event Detection and Processing

16

• Understands textual information and relations

• Generates a semantic knowledge network

• Identifies market moving news in real-time

… big launch celebrations at hardware stores with Galaxy Tab III were canceled. Apple sues Samsung in Australia. Following earlier legal disputes …

Apple

sues Samsung

in Australia

ACTING COMPANYNEGATIVE RELATIONRECEIVING COMPANY

LOCATION OF RELATION

legal action Samsung

Microsoft

Apple

Sony

Nokia

Motorola

Sharp

China

Rare Earths

Foxconn

Clueda AG

Event Determination With Big Data Analytics

17

t0 t1

Price

Time

open

low

close = high

News Release

market move

move causedby news

measurementerror

Clueda AG

Analysis Of News From One Year

18

Number of news

Thresholdmarket move

Meaningfulnews events

Optimalthreshold

Event Type 2

Event Type 1

Clustering

Clueda AG

Event Types

19

Event Rel Freq.

CDS Price Move 1

Analyst Forecast 1

Business Climate Change 1

CEO Search 1

Company Forecast 1

Customer Problems 1

Debt Financing 1

Equity Financing 1

ErrorSymbolAssignment 1

Fraud Investigation 1

Government Decision (no bailout) 1

Incorporation Change 1

Legal Settlement 1

M&A 1

Restructuring 1

Supply Chain 1

Trading Halt 1

Asset Liquidation 2

Stocks Fall (Peers) 2

Dividend Change 3

Broker Rating 9

Quarterly Results 10

Clueda AG

Statement-Centric Information Compression and Detection

Approximately 30-40% of all news contain redundant information

Only one out of 500 news is market moving

20

Clueda AG

Identify Relevant Information from Noise

21

Clueda AG

Behavioural Finance

22

“We find an accuracy of 87.6% in predicting the daily up and down changes in the closing values of the DJIA”

Clueda AG

Sentiment Detection

Simple approach: Counting positive and negative words

Problems

23

Clueda AG

Systemic Interrelations / Systemic Mood

24

Samsung

Microsoft

SonyNokia

Motorola

Sharp

China

Rare Earths

Foxconn

Apple

Foxconn

SonyNokia

Motorola

Sharp

legal actionSamsung

• Sentiment influences with systems biological methods

• Mood propagation in networks• Identification of indirect mood

drivers

Clueda AG

Sentiment works in multi factor models

25

Clueda AG

Understanding Complex Situations

Extraction from networks with millions of nodes and billions of edges

26

Clueda AG

Semantic Big Data News Analytics

Big Data is a reality

Big Data pitfalls

Junk in – Junk out

Correlation vs. Causation

Combination with intelligent methods is mandatory

Semantic analysis

Network analysis

It works

27

“Wir sparen mit der Software jeden Tag Tausende von Euros”

Uto Baader - Baader Bank

Clueda AG

Thank You!

Volker Stümpflen

Michael Schramm

28