Big Data for Social Good

Big Data for Social Good Using Kaggle for Business and Social Impact

Peter Prettenhofer @datarobot

Tobias Pfaff @datalook

Emmanuel Letouzé @datapopalliance

Agenda

Big Data for Social Good

> The Macro Perspective

> The Micro Perspective

…and Kaggle

> Short Intro

> Interactive Crash Course

Interactive part: 1. Load bit.ly/bitkom15 and create account 2. Create account on kaggle.com

Big Data and Development

> The Macro Perspective

Emmanuel Letouzé

rce: Oxfam

Intern

ational, citin

g Credit Su

isse, Jan. 20

“The bottom half of the world’s population owns the same as the richest 85 people in the world”*

Emmanuel Letouzé

Big Data as / is a new ecosystem: from the 3 Vs to the 3Cs of Big Data

i. Exhaust

ii. Web

iii. Sensing

Crumbs

Capacities

Community

Emmanuel Letouzé

Applications: Taxonomy

1. Descriptive -e.g. maps, clouds..

2. Predictive: -forecasting -inference

3. Prescriptive -causal inference

Emmanuel Letouzé

Applications: ongoing

Emmanuel Letouzé

Applications: Examples

Emmanuel Letouzé

Implications: Ethics

Emmanuel Letouzé

Implications: Power

Jonathan Glemmie, The Guardian, Oct 3, 2013

Emmanuel Letouzé

Tobias Pfaff

Bring the superpowers of data science to nonprofit organizations and to the local administration.

Tobias Pfaff

Saving lives with predictive analytics in New York City

Tobias Pfaff

[Slide from Drew Conway, http://bit.ly/1BjFpvW] Tobias Pfaff

Time that buildings are at risk of severe fire significantly reduced.

Which social problems can be solved with (Big) Data?

Tobias Pfaff

Who are the other players?

Tobias Pfaff

…and how can my company get involved?

Tobias Pfaff

[http://bit.ly/1w6fccS] Tobias Pfaff

> Short Intro to Data Science Competitions with

Peter Prettenhofer

How Data Science Competitions Work Return% ProductID Dept Price MFR

1.94 54323 Household 54.95 USA0.023 92356 Household 9.95 USA

0.8 78023 Computer 4.5 China0.01 12340 Audio 109.99 China0.41 31240 Audio 29.99 Taiwan0.97 12351 Hardware 54.95 Mexico

0.0115 90141 Hardware 4.99 USA0.4 81240 Hardware 6.55 Taiwan0.03 14896 Computer 211.99 Korea

0.205 62132 Computer 1100 USA1.6878 54323 Audio 34.99 USA0.0345 92356 Audio 7.99 USA

0.64 78023 Household 229.9 Brazil0.72 12340 Audio 19.95 Mexico0.41 31240 Computer 6.99 Taiwan1.94 54323 Hardware 11.99 Taiwan

0.023 92356 Household 2.05 USA0.08 78023 Computer 99.99 USA2.09 12340 Computer 129.99 China1.1 31240 Audio 18.99 China

Target Features

Peter Prettenhofer

How Data Science Competitions Work Return% ProductID Dept Price MFR

1.94 54323 Household 54.95 USA0.023 92356 Household 9.95 USA

0.8 78023 Computer 4.5 China0.01 12340 Audio 109.99 China0.41 31240 Audio 29.99 Taiwan0.97 12351 Hardware 54.95 Mexico

0.64 78023 Household 229.9 Brazil0.72 12340 Audio 19.95 Mexico0.41 31240 Computer 6.99 Taiwan1.94 54323 Hardware 11.99 Taiwan

0.023 92356 Household 2.05 USA0.08 78023 Computer 99.99 USA2.09 12340 Computer 129.99 China1.1 31240 Audio 18.99 China

Training

Testing Ground Truth

Peter Prettenhofer

How Data Science Competitions Work

Training

Testing Blanks

Return% ProductID Dept Price MFR1.94 54323 Household 54.95 USA

0.023 92356 Household 9.95 USA0.8 78023 Computer 4.5 China0.01 12340 Audio 109.99 China0.41 31240 Audio 29.99 Taiwan0.97 12351 Hardware 54.95 Mexico

? 78023 Household 229.9 Brazil? 12340 Audio 19.95 Mexico? 31240 Computer 6.99 Taiwan? 54323 Hardware 11.99 Taiwan? 92356 Household 2.05 USA? 78023 Computer 99.99 USA? 12340 Computer 129.99 China? 31240 Audio 18.99 China

Peter Prettenhofer

Training

Testing Submissions

0.83 78023 Household 229.9 Brazil0.65 12340 Audio 19.95 Mexico0.52 31240 Computer 6.99 Taiwan1.74 54323 Hardware 11.99 Taiwan0.1 92356 Household 2.05 USA0.02 78023 Computer 99.99 USA2.9 12340 Computer 129.99 China0.83 31240 Audio 18.99 China

Peter Prettenhofer

Training

Testing Public/Private

Leaderboard

0.83 78023 Household 229.9 Brazil0.65 12340 Audio 19.95 Mexico0.52 31240 Computer 6.99 Taiwan1.74 54323 Hardware 11.99 Taiwan0.1 92356 Household 2.05 USA0.02 78023 Computer 99.99 USA2.9 12340 Computer 129.99 China0.83 31240 Audio 18.99 China

Peter Prettenhofer

> Interactive Kaggle Crash Course

Peter Prettenhofer

Interactive part: 1. Load bit.ly/bitkom15 and create account 2. Create account on kaggle.com Slides and interactive material available at bit.ly/bitkom16

Thank you.

Peter Prettenhofer peter@datarobot.com

Tobias Pfaff tobias@datalook.io

Emmanuel Letouzé eletouze@datapopalliance.org

Questions?

Big Data for Social Good

Data & Analytics

Big Data What is Big Data? Recently much good science, whether physical, biological, or social, has been forced to confront - and has often benefited from

Big Blue Social

Big Sports Have Big Environmental and Social Consequences

Big Data - for good?

Big Data for Social Good

Social Studies Unit 3: “Big Business and Big Cities” Social Studies Unit 3: “Big Business and Big Cities”

Harnessing Big Data for Social Good: A Grand Challenge for Social

Social Engagement For Social Good

Scaling Big Data for Social Good: The need for sustainable … · 2018-12-17 · 1 SAIG BIG DATA SIA GD Contents Executive Summary 2 Introduction 4 1. Context: SDGs and big data 5

3. Technology Trends Social Big Data3.2.1 Properties of Social Big Data Comparing social big data with factual big data from the perspective of big data analysis, its unique properties

Big Ideas win. Good ads don't

I Feel Good-Full Big Band

Big Social Data: The Social Turn in Big Data

Social Media, Social Good

Social Media for Social Good

Big Social - SogetiLabsblog.vint.sogeti.com/.../Big-Social-Predicting-Behavior...Big-Data.pdf · Big Social Predicting behavior with Big Data ... 8 Social media as lens distortion

Good chi social for good

of Social Work & Social Welfare · computational social science HARNESSING BIG DATA FOR SOCIAL GOOD IS A COMPELLING CHALLENGE We live in a digital society in which vast quantities

Big Social Mobile

BIG AND OPEN DATA FOR SOCIAL GOOD