Guest Lecture

@soody or

linkedin.com/in/sureshsood

24 August 2014

“Wake me up tomorrow has already happened !”Big Data, Analytics, IOT and Foresight

Topic Areas

1. Quick History of Foresight

2. Social media and predictive capabilities

3. New and innovative information sources

4. Internet of Things

5. Systems of Insight

100 Years of Foresight Writing

PESTEL AnalysisNarrative & Story

ScenariosTrend Spotting/Ethnography

ForecastingEconometrics

Historical AnalysisPatent Analysis & Tracking

Global Strategic Trends out to 2045UK Ministry of Defence, 15 July 2014

www.bit.ly/Global2045

https://www.gov.uk/government/publications/global-strategic-trends-out-to-2045

Twitter and Marketing Predictions• Tweets is “found data” without asking questions

• More meaning than typical search engine query• • Large numbers of passive participants in natural settings

• Twitter can predict the stock market (Lisa Grossman, Wired, Oct 19 2010)

• Predict movie success in first few weekends of release– “…it also raises an interesting new question for advertisers and marketing

executives. Can they change the demand for their film, product or service buy directly influencing the rate at which people tweet about it? In other words, can they change the future that tweeters predict?”

Tech Review, http://www.technologyreview.com/blog/arxiv/25000/

7

Detecting flu trends using search engine query data (intentionality)

8

https://www.recordedfuture.com/

Web is Loaded with Events

Silicon Valley executives head to Vail, Colo. next week for the annual Pacific Crest Technology Leadership Forum

The carrier may select partners to set up a new carrier as early as next month

“2010 is the year when Iran will kick out Islam. Ya Ahura we will.”

“... Dr Sarkar says the new facility will be operational by March 2014...”

Drought and malnutrition hinder next year’s development plans in Yemen...

“...opposition organizers plan to meet on Thursday to protest...”

“Excited to see Mubarak speak this weekend...”

“According to TechCrunch China’s new 4G network will be deployed by mid-2010”

“Strange new Russian worm set to unleash botnet on 4/1/2012...”

https://www.recordedfuture.com/

From Keywords to Timelines

Timelinethe World/Web

“Record what the world knows about the future”

https://www.recordedfuture.com/

Recorded Future Architecture

70,000 Real-time Sources

3+ Billion Time-tagged Facts

100,000 future events/day

https://www.recordedfuture.com/

Data Types • Astronomical • Documents • Earthquake• Email• Environmental sensors • Fingerprints• Health (personal) Images• Location• Marine• Particle accelerator Satellite• Scanned survey data Social media• Sound• Text• Transactions• Video

New Sources of Information (Big data) : Social Media + Internet of Things Innovations

7,919 40,204

2,003,254,102 51 Gridded Data Sources

The ANZ Heavy Traffic Index comprises flows of vehicles weighing more than 3.5 tonnes (primarily trucks) on 11 selected roads around NZ. It is contemporaneous with GDP growth.

The ANZ Light Traffic Index is made up of light or total traffic flows (primarily cars and vans) on 10 selected roads around the country. It gives a six month lead on GDP growth

http://www.anz.co.nz/commercial-institutional/economic-markets-research/truckometer/

Useful References Informing our Thinkingon Mobility and Movement

(Silva et al (2013) A comparison of Foursquare and Instagram to the study of city dynamics and urban social behavior, Proceedings of the 2nd ACM SIGKDD International Workshop on Urban ComputingInstagram and Foursquare datasets might be compatible in finding popular regions of cityChaoming Song, et al. (2010), Limits of Predictability in Human Mobility, Science There is a potential 93% average predictability in user mobility, an exceptionally high value rooted in the inherent regularity of human behavior. Yet it is not the 93% predictability that we find the most surprising. Rather, it is the lack of variability in predictability across the population.Scellato et al. (2011), NextPlace: A Spatio-temporal Prediction Framework for Pervasive Systems. Proceedings of the 9th International Conference on Pervasive Computing (Pervasive'11)Daily and weekly routines => Few significant places every day => Regularity in human activities => Regularity leads to predictability

Domenico, A. Lima, Musolesi.M. (2012) Interdependence and Predictability of Human Mobility and Social Interactions. Proceedings of the Nokia Mobile Data Challenge Workshop.we have shown that it is possible to exploit the correlation between movement data and social interactions in order to improve the accuracy of forecasting of the future geographic position of a user. In particular, mobility correlation, measured by means of mutual information, and the presence of social ties can be used to improve movement forecasting by exploiting mobility data of friends. Moreover, this correlation can be used as indicator of potential existence of physical or distant social interactions and vice versa. Sadilek, A and Krumm, J. (2012) Far Out: Predicting Long-Term Human MobilityWhere are you going to be 285 days from now at 2pm …we show that it is possible to predict location of a wide variety of hundreds of subjects even years into the future and with high accuracy.

Useful References Informing our Thinkingon Mobility and Movement

Insights-driven businesses will generate $1.2 trillion in 2020

Forrester Research, 2016

Reports&

Analysis

Visualisation&

Interpretation

WriteData/Business

“Story” Insights

Led by Data Analyst or Scientist

Data Aggregation Operationalise

Detect & ExtractPatterns andRelationships

Generate Insights &

Story

ProcessApplication

IoT

Data Aggregation or

Data Set

Traditional Analytics: Slow & Expensive80% of time sifting through data

System of Insight (SoI)

SoI: Fast & Cost Effective80% of time in decision making with client

Demonstration of NaturalLanguage Generation ofStories from data as wellas automated pattern extraction from data

Actionable Insights

1. What now ?

2. So what ?

3. Now what ?

Companies are reimagining Business Processes with Algorithms and there is “evidence of significant, even exponential, business gains in customer’s customer engagement, cost & revenue performance”

Wilson, H., Alter A. and Shukla, P. (2016), Companies Are Reimagining Business Processes with Algorithms, Harvard Business Review, February, https://hbr.org/2016/02/companies-are-reimagining-business-processes-with-algorithms

In the News

Augmentation Steps for Advisors Using CA Kairos (adapted from Davenport, 2016)

Better customer experiences . . .. . . and half the inventory-carrying costs of other online fashion retailers.

Forrester, 2016

Systems of Insight

• Automated pattern extraction

• Outlier detection

• Correlation

• Time series

• Analytics integration with process, app or IoT

https://ubereats.com/melbourne/

Forrester 2016

Systems of Insight

• Helps move away from “crisis levels” in talent

• Traditional 5 step analytics process reduced to 2 step from data to action

• Reimagine business processes through “machine engineering”

• Minimise messy data issues and data preparation time

The future is impossible to predict. However one thing is certain :

The company that can excite it’s customers dreams is out ahead in the race to business success

Selling Dreams, Gian Luigi Longinotti

Coca-Cola Amatil invests in data sciencehttp://www.itnews.com.au/news/coca-cola-amatil-invests-in-data-science-431153

“CCA have a very good historical dataset - we collect lots of data from a variety of sources including manufacturing, trucks, fridges, and from customers… The main target challenge … is to be more proactive. [We] want to predict where the market wants to go and the trend of the market in the future.”

Former CBA data scientist Siamak Tafavogh