61
Big Data

Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Big Data

Page 2: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

1990’s “Big Data”— John Mashey, Silicon Graphics

Page 3: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

1990’s Where is the wisdom we have lost in knowledge?Where is the knowledge we have lost in information?

— T.S. Eliot1924

Page 4: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

1990’s A vast confusion of of vows, wishes, actions, edicts, petitions, lawsuits, pleas, laws, proclamations, complaints, grievances are daily brought to our ears. New books every day, pamphlets, currantoes, stories, whole catalogues and volumes of all sorts, new paradoxes, opinions, schisms, heresies, controversies in philosophy, religion, etc.

— Robert Burton

1924

1621

Page 5: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

1990’s Even if all knowledge could be found in books...it would take longer to read those books than we have to live in this life and more effort to select the useful things than to find them oneself

— René Descarte

1924

16211600

Page 6: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

1990’s Ars longa, vita brevis.(Life is short; art is long)

— Hippocrates1924

16211600

400 B.C.

Page 7: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

2009 Ars longa, vita brevis.(Life is short; art is long)

— Hippocrates1924

16211600

400 B.C.

Long Data

Page 8: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Data Has Always

Been BigKyle Erf

Page 9: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

History?

Page 10: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

History gives us a place.

Page 11: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

History gives us a place.History shows us patterns.

Page 12: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

History gives us a place.History shows us patterns.

History shows us our situation is not inevitable, we have many more

options than we realize.

Page 13: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Information has always outpaced our means of comprehending it.

Comprehending it always creates more information.

Page 14: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

The Beginning

Page 15: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

2.5 Million BC

Page 16: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

2.5 Million BC 70,000 B.C.

Page 17: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Too many people to know!

Page 18: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Myths

Page 19: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

MythsTrust

Trade

Coordination

Society

Page 20: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 21: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 22: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

2.5 Million B.C. 70,000 B.C.10,000 B.C.

Page 23: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

2.5 Million B.C. 70,000 B.C.10,000 B.C.

Page 24: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Too many amounts to keep in our heads!

Page 25: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Writing

Page 26: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 27: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 28: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

3000 B.C.

Page 29: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 30: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 31: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

The case against writing

Page 32: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 33: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

“[Writing] will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory...You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant and hard to get along with.”— Socrates

“sloppy learning” — Zhu Xi

Page 34: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

The side-effects of writing

Page 35: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 36: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 37: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 38: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

The Dark Ages?

Page 39: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Nope, there’s still too much data!

Page 40: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 41: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 42: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 43: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 44: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 45: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 46: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 47: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Math is tedious!

Page 48: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Logarithms

Page 49: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

5231 * 6772103.718584720 * 103.8307169

10(3.718584720 + 3.830716949)

107.5493016

35,424,332

Page 50: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 51: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 52: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 53: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 54: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

1890 Census!

Page 55: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Too many people to know!

Page 56: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 57: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 58: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 59: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to
Page 60: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

The cycle continues...

Page 61: Big Data - USENIX · Big Data . 1990’s “Big Data ... Myths ust de on ety. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. 2.5 Million B.C. 70,000 B.C. 10,000 B.C. Too many amounts to

Further ReadingJames Gleick - The InformationAnn Blair - Too Much To Know

James Burke - Connections