Data needs to be analysed: The Social and Economic impact of Big Data

Los datos hablan. El impacto de Big Data en la Sociedad y la Economía

Dr Pedro A. de AlarcónSenior Data Scientist. External Positioning and Big Data for Social Good

@LUCA_D3

1. Let Data Speak

2. ¿Qué está haciendo Telefónica?

3. Algunas ideas para durante y después del Máster

4. … y un buen ejemplo.

@LUCA_D3

@LUCA_D3@pdealarcon

¿Qué está pasando?... Adapt or Die

@LUCA_D3

@LUCA_D3

Let Data Speak01

Are you listening?

@LUCA_D3

Using CDR traffic to understand the impact of an earthquake in Mexico

Are you listening?

@LUCA_D3

Using sports analytics software to understand footballer behaviour

Xavi: movements faster than 5 m/sec

Messi: movements faster

than 5 m/sec

Time at each position sec.

Big Data for Football Analytics

@LUCA_D3

Time at each position sec.

Open Data for Customer Benefit

@LUCA_D3

Using open data to provide automated Oyster refunds in London

Open Data for Marketing and Brand Awareness

@LUCA_D3

Using open data as an input for a real-time advertising campaign in London

Big Data is no longer an emerging technology…

@LUCA_D3

“We’ve retired the Big Data hype cycle. I know some clients may be really surprised by that because Big Data was a really important one for many years.” Betsy Burton, Gartner

Tools and Infrastructure

@LUCA_D3

Difficulty

Val

ue

Descriptive Analytics

Diagnostic Analytics

Predictive Analytics

Prescriptive Analytics

What happened?

Why did it happen?

What will happen?

How can we make it happen?

Source, Gartner 2012

Volume Velocity

Variety

3 Vs of Big

DataTerabytesRecords

TransactionsTables / Files

BatchReal-timeStreams

Near-time

StructuredUnstructured

Semi-structuredAll of the above

Data Science

@LUCA_D3

“Data Scientist is the sexiest job in the 21st century.”

“Big Data expertise demand has increased by 85% in the last year.”

“21.000 Data Engineer jobs posted online.”

“90.000 Data Engineer jobs posted online.”

@LUCA_D3

¿Qué está haciendoTelefónica?02

We have launched LUCA Data-Driven Decisions

@LUCA_D3

Our goal: to empower other organizations to become more data-driven

The Big Data journey has different maturity stages

@LUCA_D3

• Interested in understanding potential of Big Data

• Keen to run POCs

• Lack of internal know-how and capabilities

• Data sources unknown / uncontrolled

• Strategy and uses cases not defined

Exploration

Data-Driven

• Big Data already implemented in a “data-centric” organization

• Data is intrinsic in decision-making

• Data brings organization closer to its customers

• In-house expertise in data science and data engineering

• External data sources used regularly to generate richer insights

• New data-driven business models and processes in creation

Transformation

• Decision made to leverage Big Data

• Require methodology and expert resources

• Need technology and infrastructure

• Analytics capabilities to be deployed / outsourced

• End-to-end transformation programme planned, including cultural and org. changes

And this is Telefonica’s Big Data journey – so far

@LUCA_D3

2011 20132012

Explorationproving Big Data’s

potential

Smart Steps launched

SNA pilots marketing and churn

Global BI organization created

Data to become an asset

20152014

Transformationaccelerating adoption

Smart Digits launched

China Unicom Joint Venture

Smart Steps International roll out

Key use cases executed

Strategic data sourcing installed

20172016

Data-Drivendemocratizing Big Data

Big Data B2B unit launch

Niche Big Data consultancy acquired

Cultural transformation

CDO (Chema Alonso) appointed

Breaking all kinds of silos

Big Data roadmap for each Operating Business

New data-driven P&S

These are the components to work on during the journey

@LUCA_D3

Data

Engineering

Tools &

Infrastructure

Data

Science

Business Insights

Strategy and Transformation (Strategic Assessments, Corporate Transformation Programmes, Culture and Talent programmes)

End 2 End Security Wrap

Data Engineering – more data sources

@LUCA_D3

Big DataData Warehouse

+ External DataGeolocation

Call Centre Content

Channels (online & offline)

TV Audiences

Q of E

Sociodemographic

Products and Services

Consumption

Customer

Web LogsDescriptive

WHO ?

Behavioral WHAT, WHERE ?

InteractionHOW ?

AttitudinalWHY ?

All of which is applied to the Business

@LUCA_D3

Product and Innovation

Operational and Finance

Channels

Sales and Marketing

Network and Infrastructure

Business optimisation

which have allowed us to create value internally:

@LUCA_D3

Smart Marketing

Churn Reduction & Optimisation

Network Planning

Optimization

Real Time contextual campaigns

Video analytics

AData-Driven

Online channel

E2E churn approach

Multi-channel analytics

Fibre/LTE deployment

Supply/demand analysis

Value based analysis

Procurement analytics

Fleet management optimisation

Business Insights for Sustainable transport

@LUCA_D3

Using mobile data to measure the sustainability of commuting in Barcelona

Business Insights for Tourism Insights

@LUCA_D3

Using mobile data to understand the catchment and profile of tourists to the Fallas Festival

Business Insights for Tourism Insights

@LUCA_D3

Using mobile data to understand the catchment and profile of tourists to Spain

Business Insights for Smarter Communication

@LUCA_D3

Using analytics to optimize global communications in multinationals

Big Data for Social Good

@LUCA_D3

Using CDR traffic to understand the impact of floods in Mexico

@LUCA_D3

Algunas ideas03

Aprovechad las clases presenciales

@LUCA_D3

Vs.

Leed blogs de Ciencia de Datos

@LUCA_D3

… os ayudarán a averiguar cuales son vuestrosverdaderos intereses

Expresad vuestros intereses y mostrad vuestro valordiferencial ANTES del primer trabajo… o como escribirun CV aplastante sin usar Word.

@LUCA_D3

• Kaggle• Github• Ipython Notebooks• Hackatons• Open data Apps (Code4Good)• StackOverflow• Blog• Twitter & Linkedin

Los datos deben contar una historia: contadla bien.

@LUCA_D3

http://www.forbes.com/sites/brentdykes/2016/03/31/data-storytelling-the-essential-data-science-skill-everyone-needs/#1098ffadf0c8

No olvideis las herramientas comerciales

@LUCA_D3

Cambio de enfoque

@LUCA_D3

Teoría Práctica

ValidaciónCaso de uso/ Problema

Preparaciónde datos Teoría

ProgramaciónModelos /

Visualización

Industrialización

Máster

Empresa

Si Mahoma no va a la montaña…

@LUCA_D3

• Lenguajes & Machine Learning: Python, R, Scala

• Big Data (básico): Spark, Hive.

• Demandas no bien cubiertas: Modelos Predicitivos en Real Time.Visualizaciones impresionantes (D3)Mapas (PostGis)

• Vienen fuerte: Deep LearningTensor FlowAI+NLP (Chatbots)

@LUCA_D3

Un buen ejemplo ;-)04

@LUCA_D3

QUE ESPERABA…

• Conocer estructura de una gran empresa

@LUCA_D3

QUE ESPERABA…

• Tratar con datos reales y muy diferentes

@LUCA_D3

QUE ESPERABA…• Grandes proyectos

@LUCA_D3

QUE ME HE ENCONTRADO…

• Una gran empresa

@LUCA_D3

QUE ME HE ENCONTRADO…• Aprendizaje teórico continuo

SERIES TEMPORALES

ANÁLISIS DE REDES

ESTADÍSTICA

PREPROCESAMIENTO

@LUCA_D3

QUE ME HE ENCONTRADO…

• Aprendizaje tecnológico continuo

@LUCA_D3

EJEMPLOS …

• GLOBAL RIDER

https://www.youtube.com/watch?v=XeCTZcHqqyc

• ELECCIONES ESTADOS UNIDOS

http://data-speaks.luca-d3.com/2016/11/big-data-and-elections-we-shine-light.html

Global Rider

@LUCA_D3

https://www.youtube.com/watch?v=XeCTZcHqqyc

@LUCA_D3

Elecciones USA 2016 & Twitter

http://data-speaks.luca-d3.com/2016/11/big-data-and-elections-we-shine-light.html

TwitterSinfonier(11Paths) MongoDB

PhilipsHue + Spotfire

RealTime Filter Python

Elastic + Kibana

vs

@LUCA_D3

Elecciones USA 2016 & Twitter

http://data-speaks.luca-d3.com/2016/11/big-data-and-elections-we-shine-light.html

@LUCA_D3

Let your data speak

Thanks for listening!@vrbenjamins / @LUCA_D3

www.luca-d3.com