Embed Size (px)
Citation preview
USING AI AND BIG DATA TO SOLVE
CORE INVESTMENT PROBLEMS
Larry Cao, CFA
Senior Director, Industry Research
6 November 2019 | CFA VBA Netherlands
OVERVIEW
2
01 What can AI and big data do?
02 How should we respond?
03 What are the best practices in applying AI/ big data in investments?
RELATED CFA INSTITUTE PUBLICATIONS
3
4
WHAT CAN AI AND BIG DATA DO?
ARTIFICIAL INTELLIGENCE (AI) INTO
LIVING ROOM
5
10
20
30
40
50
60
70
80
90
100
Se
p-1
4
Se
p-1
5
Se
p-1
6
Se
p-1
7
Se
p-1
8
Se
p-1
9
ARTIFICIAL INTELLIGENCE: (WORLDWIDE)
AI: THE HYPE CYCLE
6
Source: Google Trend
AI AND BIG DATA ENABLE HUMAN BEINGS TO
7
PROCESS
NEW DATA
that we did not have
access to or were not
able to process before
PROCESS DATA
IN NEW WAYS
that we were not
able to before
ARTIFICIAL INTELLIGENCE
NLP, COMPUTER VISION, VOICE RECOGNITION
8
NLP
Natural Language
Processing
COMPUTER VISION
Image processing
VOICE RECOGNITION
Turning voices or spoken
language into text
WHAT PROGRESS HAVE AI
RESEARCHERS MADE?
9
COMPUTER VISION
In the ImageNet competition
of 2017, AI programs beat
the best human record by
an increased margin
MACHINE LEARNING
In January 2018,
Google debuted the
Cloud AutoML platform
NLP
Two AI programs have
succeeded in reading better
than an average adult as of
January 2018
VOICE RECOGNITION
Last year, Google and
Microsoft speech
recognition programs
transcribed as accurately
as humans
Source: literature search
ALTERNATIVE AND UNSTRUCTURED DATA
10
ALTERNATIVE DATA
(not currently used)
UNSTRUCTURED DATA
(not readily processable)
ARTIFICIAL INTELLIGENCE, MACHINE
LEARNING, AND DEEP LEARNING
11
ARTIFICIAL INTELLIGENCE
Computers that can see, hear, and
understand humans.
MACHINE LEARNING (ML)
Field of study that gives computers the ability
to learn without being explicitly programmed
DEEP LEARNING (DL)
Multi-layer neural networks
BENEFITS OF AI & BIG DATA
12
NEW DATA:
More thorough analysis
for analyst
Type I
NLP
Computer Vision
Voice Recognition
Type II Big data
NEW WAYS TO
PROCESS DATA:
Better informed
decision for PMs
Type III ML / DL
AI CHANGING FINANCE
13
MANAGER MONITORING HEALTH SCREENING
BIG DATA CHANGING FINANCE
14
INTELLIGENT PRICING PRODUCTION ESTIMATION
CFA INSTITUTE SURVEY ON AI
Jan/Feb 2019
15
28
79
12
40
208
22
124
0
50
100
150
200
250
EquitySell-SideAnalyst
EquityBuy-SideAnalyst
CreditSell-SideAnalyst
CreditBuy-SideAnalyst
PortfolioManager
ChiefInvestment
Officer
PrivateWealth
Manager
Profile of Survey Respondents
Note: Survey participation (N=734).
10%
28%
62%>10 Years
6-10 Years
<=5 Years
B. Years of
ExperienceA. Occupation
Analyst (159) PM (230) PWM (124)
CURRENT USAGE OF AI
PM
16
Note: Survey participation (N=230).
33%
10%
19%
49%
50%
None
Artificial intelligence/machine learning to find a nonlinearrelationship or estimate
Run a backtest of an algorithm
Regression analysis to find a linear relationship
Run a backtest of a strategy
Portfolio Manager: Which of these have you used in the past 12 months for investment strategy and process?
Run a backtest of a strategy
Regression analysis to find a linear relationship
Run a backtest of an algorithm
Artificial intelligence/machine learning to find a
nonlinear relationship or estimate
None
Only 10% of the portfolio managers who responded to the survey used AI/ML
techniques to improve their investment process in the past 12 months.
CURRENT USAGE OF AI
PM
17
Note: Survey participation (N=230).
3%
3%
21%
73%
Other
Technology team initiated with specific AI/ML capabilities
Investment and technology team collaborated to selectthe ideal technology for enhancing the way an
investment task is performed
Investment team initiated the process with well-definedneeds
Portfolio Manager: Which option below most accurately describes your organization’s process
for investment strategy and process?
Investment team initiated the
process with well-defined needs
Investment and technology team collaborated
to select the ideal technology for enhancing the
way an investment task is performed
Technology team initiated with
specific AI/ML capabilities
Other
CURRENT USAGE OF AI
PM
18
Note: Survey participation (N=230).
69%
6%
8%
8%
9%
9%
9%
10%
10%
14%
15%
None
Portfolio Manager: Which of the following artificial intelligence/machine learning techniques have you performed in the past 12 months for creating trading algorithms?
None
Predicting asset price direction or finding signals from noisy data (e.g.,
using support vector machines to do supervised classification)
Identifying prevailing factors driving the market (e.g. using unsupervised machine learning,
such as principal component analysis, to determine the best representation of the data)
Finding the most profitable trading strategies (e.g.,
using reinforcement unsupervised, deep learning)
Predicting short-term asset price direction based on
macro data (e.g., using gradient boosting)
Predicting short-term asset price direction (e.g., using
lasso, k-nearest neighbor, or ridge regression)
Determining market trend or regime (e.g., using a
Hidden Markov Model supervised classification)
Examining the entire set of asset returns to identify
relationships (e.g., using unsupervised machine learning)
Determining sentiment via natural language processing of news,
Twitter, transcripts, etc. (e.g., by counting positive or negative words)
Building signals (e.g., carry signals, value signals,
technical signals, microstructure signals)
Arriving at buy or sell decisions based on macro,
fundamentals, or market input variables using classification
CURRENT USAGE OF AI
Analyst
19
Note: Survey participation (N=159).
75%
2%
6%
8%
9%
10%
14%
Other
Technology team initiated with specificAI/ML capabilities
Investment and technology teamcollaborated to select the ideal
technology for enhancing the way aninvestment task is performed
Investment team initiated the processwith well-defined needs
Run a backtest of a strategy
A. Analyst: Which of the following artificial intelligence/machine learning use cases have
you performed in the past 12 months for industry and company analysis?
Scraping third-party websites
(e.g., regulators)
Using natural language processing to read
large tracts of text, transcripts, and/or fillings
Using deep learning (e.g., long short-
term memory) to gauge sentiment in
social media and news
Extracting alpha from unstructured or
alternative data
Using robotic process automation
Using unstructured deep learning
(e.g., convolutional neural nets) to count
cars in parking lots)
None 44%
1%
11%
30%
44%
Other
Technology team initiated with specificAI/ML capabilities
Investment and technology teamcollaborated to select the ideal
technology for enhancing the way aninvestment task is performed
Investment team initiated the processwith well-defined needs
Run a backtest of a strategy
B. Analyst: What type(s) of unstructured and/or alternative data have you used for your industry and company analyses in the past 12 months?
Individual data (e.g., social media,
blogs, product reviews, web search
trends, cellphone location data)
Business data (e.g., credit card data, store
visit data, bills of lading)
Satellite data (e.g., agriculture data,
rig activity, car traffic,
ship locations, mining data)
Other
None
20
HOW SHOULD WE RESPOND?
OUR VIEWS ON EARLY STAGE FINTECH
21
Source: CFA Institute, first published in Hong Kong Economic Journal in May 2016
Fintech has been most successful in areas that are un-(under-) served
by financial institutions.
Collaboration has become the emerging theme for financial and technology
industry leaders.
“We tend to overestimate the effect of a technology in the short run
and underestimate the effect in the long run.”
OUR VIEWS ON THE ABCD OF FINTECH
22
Source: CFA Institute
AI, big data and cloud computing may transform the financial services
industry as we know it
Powerful FinTech comes from the collaboration between Fin and Tech,
which translates into, at the firm level, collaboration between powerful
financial institutions and tech giants and, at the individual level, team
development that focus on the collaboration between finance and
technology talents.
“We tend to overestimate the effect of a technology in the short run
and underestimate the effect in the long run.”
COLLABORATION BETWEEN F.I. AND TECH IN AI
23
Source: Literature search
HAVEN BY BERKSHIRE
The Deal with Amazon
24
THE NEW
HEALTH INSURANCE
COMPANY
THE FINTECH PYRAMID
25
COST
TALENT
TECHNOLOGY
VISION
TIME
FIN TECH
FINTECH
BUILDING T-SHAPED TEAM
26
INVESTMENT
TECHNOLOGY
INNOVATION
INVESTMENT
• Investment decision maker
• Investment researcher
• Private wealth manager
TECHNOLOGY
• Data scientist
• Application engineer
INNOVATION
• Investment thinking and
process innovator
• Knowledge engineer
• Innovation facilitator
SIGNIFICANT TYPES OF
ROLES AT INVESTMENT
FIRMS OF THE FUTURE
WORD OF CAUTION
AI and Big Data Are No Panacea
27
MAN + MACHINE OR MAN VS. MACHINE?
28
AI+
HI+
AI-
HI+
AI+
HI-
AI-
HI-
29
WHAT ARE THE BEST PRACTICES IN
APPLYING AI/ BID DATA IN
INVESTMENTS?
MAN AHL
30
Long–only and alternative
investment management firm
> USD$110B assets under
management as of June 2019
First started researching
ML and its application in
investments in 2009,
ML strategy entered program
portfolio in 2014
ASSET
ALLOCATIONEQUITY DEBT HEDGE FUNDS
AMERICAS
ASIA PACIFIC
EUROPE, MIDDLE
EAST, AND AFRICAX
AREAS WHERE ML HAD
THE MOST IMPACT
• Developing trading
strategies
• Improving efficiency
of execution
TEAM
FORMATION
• Researchers: Scientific
background
• ML team: non-financial
backgrounds + research
experience
• Integrated team, no clear
distinction between
researchers, data
scientists, and portfolio
managers
MAIN ML
TECHNIQUES USED
• Bayesian ML, DL
• Pattern recognition
algorithms
• Strategies based on NLP
ROLES IN THE
DEVELOPMENT PROCESS
• CIO, senior members:
Identify main directions
• ML team, rest of
research: Capture
trading signals
MAN AHL
ML in Trading Strategies/Execution
31
MAN AHL
32
EXPERIENCE
MATTERS
Only a fleetingly
small percentage of
data is “useful”
EMBRACE OPEN
SOURCE
Stay involved over the long
term by contributing back.
Form a virtuous circle
BE BOLD IN THE
PROCESS
Have resolve to decide
what is worth pursuing,
and kill off projects that
don’t look promising
Our maxim is, therefore, “Use the simplest tool that does the job.”
KEY TAKEAWAYS
NEW YORK LIFE INVESTMENTS
33
Top US asset
management firm
> US$300B AUM in 2018 Multi-Asset Solutions
team manages US$10B
in global macro asset
allocation products
ASSET
ALLOCATIONEQUITY DEBT HEDGE FUNDS
AMERICAS X
ASIA PACIFIC
EUROPE, MIDDLE
EAST, AND AFRICA
NEW YORK LIFE INVESTMENTS
Smart Analysis Gives Clearer Picture
34
• Focus on most
important indicators
• Better assess risks
and opportunities
Market Drivers
e.g. Economic
cycle
Analyze
by ML
Cycle Trends
Interpretation
e.g. Volatility
RISK FACTORS
CYCLE MOMENTUM SENTIMENT VALUE
NEW YORK LIFE INVESTMENTS
35
ML techniques have enabled us to
incorporate larger volumes of data,
improve the accuracy of the
predictions, and identify the most
important predictors to monitor
in dashboards.
The signals generated from ML
techniques—especially the cycle and
value signals—help us focus on the
most important indicators. The
cycle framework has also allowed us
to monitor a wider range of indicators.
KEY TAKEAWAYS
GOLDMAN SACHS
36
Global Investment Research Data
Strategy team works with equity and macro
research analysts on projects that require
analytical and quantitative skill sets
Collaborated on nearly 200
published research analyses across various
sectors and markets worldwide
ASSET
ALLOCATIONEQUITY DEBT HEDGE FUNDS
AMERICAS X
ASIA PACIFIC
EUROPE, MIDDLE
EAST, AND AFRICA
ESTIMATING A QUARRY’S MARKET SHARE
37
GEOSPATIAL
LIBRARIES
QUARRY SPECIFIC
METADATA
QUARRIES
LOCATION
AGGREGATE
PRODUCTION
GOLDMAN SACHS
Leveraging AI/Alternative Data Analysis in Sell-Side Research
38
THREE QUESTIONS
TO ANSWER
• How to understand
positioning in a hyperlocal
industry
• How to represent market
share for public
companies in a largely
private competitive
landscape
• How to inform investment
professionals about the
directional sense of
quarterly company results
with respect to aggregate
volumes
TEAM
FORMATION
• Analyst team
• Quant research team
• Both teams are involved
with building and
validation of model
MAIN BIG
DATA USED
• Company data
• Publicly available quarry
data
INTERSECTION OF
THREE FUNCTIONS
• Tap into domain expertise
• Access all relevant
information, including
both “traditional” sources
and alternative data
where appropriate
• Apply advanced analysis
techniques to extract
relevant insights.
GOLDMAN SACHS
39
Don’t underestimate the potential of more
advanced techniques and approaches
More niche, sector-specific data sets
lend themselves much better to a
fundamental analyst or portfolio manager
KEY TAKEAWAYS
Alternative data adds to the mosaic,
but it is not a goal in and of itself
Leveraging alternative data doesn’t
necessarily mean breaking the bank
CLAMC (CHINA LIFE
ASSET MANAGEMENT)
40
ASSET
ALLOCATIONEQUITY DEBT HEDGE FUNDS
AMERICAS
ASIA PACIFIC X
EUROPE, MIDDLE
EAST, AND AFRICA
Credit management
technology service
provider
Using CreditMaster
(integrated credit
management solution
for Chinese debt
market) in 2018
AUM >USD$400B
as of 2018
China’s largest asset
management firm
CSCI (CHINA SECURITIES
CREDIT INVESTMENT)
CLAMC & CSCI
AI and Big Data Assist in Debt Portfolio Management
41
CAPABILITIES OF
CREDITMASTER
• Evaluation engine:
Generate customized
ratings (from credit risk
data + third parties’
ratings)
• Analytics module:
Integrate Creditmaster into
credit analysts and risk
managers’ workflow.
TOOLS TO GATHER AND
MONITOR RISK INFO
• Distributed web spider
system
• NLP captures information
from unstructured data
• BiLSTM (bidirectional long
short-term memory) model
• Text convolutional neural
networks
• Knowledge graph
ROLES IN THE
DEVELOPMENT PROCESS
• Multiple functions ranging from credit modeling to
systems development
• Credit analysts, quants: Develop and maintain the
methodology of credit models
• Data scientists: Collect and process data to
generate signals
• Engineers: Systems development
• Product managers: Communicate client needs to
other teams
CLAMC & CSCI
42
DEVELOPING AN INTEGRATED SYSTEM
Requires different skill sets. CSCI finds it
essential to have the five roles centralized in
one team to work toward one objective.
MONTHLY “ITERATIONS”
Receiving client feedback in a timely fashion
Reduces the risk of miscommunication between
the investment and technology functions
KEY TAKEAWAYS
SCHRODERS
43
Data Insights Unit formed in 2014 To gain an edge in data-led research
and gain early, differentiated insights into
individual companies
ASSET
ALLOCATIONEQUITY DEBT HEDGE FUNDS
AMERICAS
ASIA PACIFIC
EUROPE, MIDDLE
EAST, AND AFRICAX
SCHRODERS
Building the Data Science Team
44
STEPS TO SEARCH FOR
COMPANIES TO INVEST
• NLP algorithms to analyze article
• Clustering similar articles
• Fund manager conducts financial
analysis on interesting results
MAIN BIG
DATA USED
• ML and Bayesian inference:
harness patterns and
predictability in data
• Combination of cloud
technologies: store data
• Geospatial data
ROLES IN THE
DEVELOPMENT PROCESS
• Developing team: bringing in new
data sets and finding more use
cases
• Data Engineers: set up the
infrastructure needed to pipeline
the data
• Regular meetings between
investment professionals and
data scientists to generate and
test ideas and identify areas of
shared interest
SCHRODERS TEAM EVOLUTION
Illuminating for Firms
45
CONSTRUCT TEAM
(BEN, PM)BRING IN DATA
SCIENTIST,
CONSULTANTS
ACQUIRE EXTERNAL
EXPERTS (MARK, DATA
SCIENTIST)
DATA ENGINEERS
(FROM IT)
ANALYZE
ARTICLES
WITH AI
SCHRODERS
46
The two big lessons we have
learned are:
• To have a senior sponsor who
really believes in this sort of
innovation for support
• To have the right mix of skills
within the team
A data science capability can’t
be an exercise in doing
something fashionable for the
sake of it; it needs to add
value to the business
KEY TAKEAWAYS
Make the team a centralized
function available to
all investors
• Nobody worried about what using
the data or the team’s skills was
costing them
• This decision also set up the team
to find areas of value that spanned
multiple investment teams
SPECIAL ISSUES WITH THE
MPT EFFICIENT FRONTIER
47
ISSUES WITH THE
POPULAR MPT FRONTIER
• Unstable covariance matrices
• Unrealistic assumptions on returns
• High transaction cost
SOLUTIONS
• De-noising
• Nested clustered optimization (NCO)
ENHANCING THE MPT
EFFICIENT FRONTIER WITH ML
48
Table 1. RMSE for combinations of de-noising and
shrinkage (maximum Sharpe ratio portfolio) Table 2. RMSE for the maximum Sharpe ratio portfolio
NOT
DE-NOISED DE-NOISED
NOT SHRUNK 9.48E-01 5.27E-02
SHRUNK 2.77E-01 5.17E-02
MARKOWITZ NCO
RAW 7.02E-02 3.17E-02
SHRUNK 6.54E-02 5.72E-02
DE-NOISING WITH KERNEL
DENSITY ESTIMATOR (KDE)
NESTED CLUSTERED
OPTIMIZATION (NCO)
AI AND BIG DATA IN INVESTMENTS
Outlook
49
AI and big data have the potential to bring about the most significant change
to the investment management industry that current professionals will
experience in their careers.
Successful investment firms of the future will start to strategically plan their
integration of AI and big data techniques into their investment processes
now.
Successful investment professionals will understand and exploit the
opportunities brought about by these new technologies and applications,
enabled by collaborative organizational cultures, cognitive diversity, and
T-shaped teams.
