31
10.20.2005 All Data Big and Small May 2011

All Data Big and Small - By Stephen O'Grady

Embed Size (px)

DESCRIPTION

See conference video - http://www.lucidimagination.com/devzone/events/conferences/revolution/2011 In 2009 The Guardian launched The Open Platform, a suite of services and tools that enable content partners and developers to build applications with The Guardian’s rich content. The content API, hosted on Solr instances on EC2, contains JSON representations of all Guardian articles back to 1999 - over 1 million articles, and is an increasingly complete representation of the output of the organization. The DataStore contains curated data sets for use in applications and virtualizations. This talk will cover how The Guardian opened up their business, enriched it, and reached new markets with its Open Platform strategy. Stephen will cover the technical architecture, implementation of Solr (the key technology powering the platform), and how The Guardian has used it to embrace disruption in the media space, while finding new sources of revenue and innovation.

Citation preview

Page 1: All Data Big and Small - By Stephen O'Grady

10.20.2005

All Data Big and Small

May 2011

Page 2: All Data Big and Small - By Stephen O'Grady

2

http://redmonk.com/public/lucene.pdf

Page 3: All Data Big and Small - By Stephen O'Grady

3

In the beginning, there was the database...

Page 4: All Data Big and Small - By Stephen O'Grady

4

1979 1983 1989

Page 5: All Data Big and Small - By Stephen O'Grady

5

When you have a hammerand so on

Page 6: All Data Big and Small - By Stephen O'Grady

6

Source: http://www.flickr.com/photos/pagedooley/2234031789/

Page 7: All Data Big and Small - By Stephen O'Grady

7

December 29, 2004

Page 8: All Data Big and Small - By Stephen O'Grady

8

The Cambrian Non-relationalExplosion

Page 9: All Data Big and Small - By Stephen O'Grady

9

Source: http://www.pnas.org/content/97/9/4426/F1.expansion.html

Page 10: All Data Big and Small - By Stephen O'Grady

10

Page 11: All Data Big and Small - By Stephen O'Grady

11

Why?

Page 12: All Data Big and Small - By Stephen O'Grady

12

Different tools for different jobs

Page 13: All Data Big and Small - By Stephen O'Grady

13

Or, rather, different data

Page 14: All Data Big and Small - By Stephen O'Grady

14

A lot of different data

Page 15: All Data Big and Small - By Stephen O'Grady

15

Page 16: All Data Big and Small - By Stephen O'Grady

16

Most of the attention goes to Big Data

Page 17: All Data Big and Small - By Stephen O'Grady

17

In spite of the fact that comparatively few have it

Page 18: All Data Big and Small - By Stephen O'Grady

18

Less heralded is unstructured data

Page 19: All Data Big and Small - By Stephen O'Grady

19

Page 20: All Data Big and Small - By Stephen O'Grady

20

Between the size and (un)structure, it's amazing anything gets found

Page 21: All Data Big and Small - By Stephen O'Grady

21Source: http://www.flickr.com/photos/28705377@N04/4142872268/

Page 22: All Data Big and Small - By Stephen O'Grady

22

It's hard to ask the right question

Page 23: All Data Big and Small - By Stephen O'Grady

23

To make matters worse, you may only get one chance

Page 24: All Data Big and Small - By Stephen O'Grady

24

The most important answeris the next question

Page 25: All Data Big and Small - By Stephen O'Grady

25

Some questions

Page 26: All Data Big and Small - By Stephen O'Grady

26

Page 27: All Data Big and Small - By Stephen O'Grady

27

Page 28: All Data Big and Small - By Stephen O'Grady

28

Page 29: All Data Big and Small - By Stephen O'Grady

29

Page 30: All Data Big and Small - By Stephen O'Grady

30

Page 31: All Data Big and Small - By Stephen O'Grady

31

OTHER QUESTIONS