Transcript
Page 1: Big data - from consumers and patients, to the sea and stars
Page 2: Big data - from consumers and patients, to the sea and stars
Page 3: Big data - from consumers and patients, to the sea and stars

Ubiquitous, affordable devices and connectivity

Page 4: Big data - from consumers and patients, to the sea and stars

Big data: The next frontier for innovation, competition, and productivity

May 2011 | byJames Manyika, Michael Chui, Brad Brown, Jacques Bughin, Richard

Dobbs, Charles Roxburgh, Angela Hung Byers, McKinsey Global Institute (MGI)

Page 5: Big data - from consumers and patients, to the sea and stars

CloudBig data

Aggregation

MachineLearning Analytics

Page 6: Big data - from consumers and patients, to the sea and stars
Page 7: Big data - from consumers and patients, to the sea and stars

“Snow White Portraits”

Page 8: Big data - from consumers and patients, to the sea and stars

http://www.mypersonality.org/wiki

http://www.pnas.org/cgi/doi/10.1073/pnas.1218772110

Page 9: Big data - from consumers and patients, to the sea and stars

http://youarewhatyoulike.com/Kosinski, M., Stillwell D.J., Graepel T (2013) Private traits and

attributes are predictable from digital records of human behavior.

Proceedings of the National Academy of Sciences (PNAS).)

Page 10: Big data - from consumers and patients, to the sea and stars
Page 11: Big data - from consumers and patients, to the sea and stars
Page 12: Big data - from consumers and patients, to the sea and stars
Page 13: Big data - from consumers and patients, to the sea and stars
Page 14: Big data - from consumers and patients, to the sea and stars

Data

Acquisition &

modelling

Collaboration

and

visualisation

Analysis &

data mining

Dissemination

& sharing

Archiving and

preserving

fourthparadigm.org

Data-intensive Research

Page 15: Big data - from consumers and patients, to the sea and stars

X-Info

• Data ingest

• Managing a petabyte

• Common schema

• How to organize it

• How to reorganize it

• How to share with others

• Query and Vis tools

• Building and executing models

• Integrating data and Literature

• Documenting experiments

• Curation and long-term

preservation

The Generic Problems

Experiments &Instruments

Simulations

Literature

Other Archives

facts

facts

facts

facts

Questions

Answers

Page 16: Big data - from consumers and patients, to the sea and stars
Page 17: Big data - from consumers and patients, to the sea and stars

The Cloud

democratizes

access to scale &

economies of scale

Page 18: Big data - from consumers and patients, to the sea and stars

Genomics Revolution

Page 19: Big data - from consumers and patients, to the sea and stars

Genomics

Page 20: Big data - from consumers and patients, to the sea and stars

Manual Measurement

Automated Measurement

Sample Collection

Historical Photographs

Counting

Ubiquitous

Motes

Aircraft SurveysModel Output

Typing

Page 21: Big data - from consumers and patients, to the sea and stars

Monitoring

Collation

Quality assurance

Aggregation

Analysis

Reporting

Forecasting

Distribution

Done poorly,but a few notablecounter-examples

Done poorly to moderately,not easy to find

Sometimes done well,generally discoverable and available,

but could be improved

Integration

(I. Zaslavsky & CSIRO, BOM, WMO)

Page 22: Big data - from consumers and patients, to the sea and stars

Web search:

“open weather

data azure”

Page 23: Big data - from consumers and patients, to the sea and stars
Page 24: Big data - from consumers and patients, to the sea and stars

http://fetchclimate2.cloudapp.net/

Page 25: Big data - from consumers and patients, to the sea and stars

World Wide Telescope

www.worldwidetelescope.org http://www.worldwidetelescope.org/planck.html

Page 26: Big data - from consumers and patients, to the sea and stars
Page 27: Big data - from consumers and patients, to the sea and stars
Page 28: Big data - from consumers and patients, to the sea and stars
Page 29: Big data - from consumers and patients, to the sea and stars

www.tryfsharp.org

Page 30: Big data - from consumers and patients, to the sea and stars

Datacenter

Edge point

Page 31: Big data - from consumers and patients, to the sea and stars

http://github.com/windowsazure

Page 32: Big data - from consumers and patients, to the sea and stars

Big Data Cloud Ecosystem

Page 34: Big data - from consumers and patients, to the sea and stars

Windows

Azure

Urban

Science

Machine

Learning

Environmental

Science

Life

Sciences

Computer

science

Engineering

Page 35: Big data - from consumers and patients, to the sea and stars

British Library Labs cloud

analysis of digital catalogues,

including 19th Century books

scanned by Microsoft.@MechCuratorBot

mechanicalcurator.tumblr.com

Page 36: Big data - from consumers and patients, to the sea and stars
Page 37: Big data - from consumers and patients, to the sea and stars
Page 38: Big data - from consumers and patients, to the sea and stars

www.azure4research.com

Page 39: Big data - from consumers and patients, to the sea and stars

Use laptops &

desktop computers

Overwhelmed by

data

Finding analysis

ever more difficult;

sharing even

harder

www.azure4research.com

Page 40: Big data - from consumers and patients, to the sea and stars
Page 41: Big data - from consumers and patients, to the sea and stars
Page 42: Big data - from consumers and patients, to the sea and stars
Page 43: Big data - from consumers and patients, to the sea and stars

Recommended