Working With Data and Humans

Working With Data and Humans

Daniel X. O’Neil@juggernautco

@juggernautco

Me

• Daniel X. O’Neil• Co-founder of EveryBlock• 2007 Knight News Challenge• Executive Director of Smart Chicago

Collaborative• 2012 Knight Community Information

Challenge

@juggernautco

The Data Revolution

• I know about some, but not all of it• Since about 2005• Working with the Mayor’s Office in Chicago• ChicagoWorksForYou.com• Then at EveryBlock, where I was responsible

for data acquisition

@juggernautco

The Data Revolution

• 8 Principles of Open Government Data• Independent Government Observers Task

Force• POTUS Executive Orders on Inauguration Day• Apps contests• Municipal ordinances• Socrata• Code for America

@juggernautco

There’s Data and There’s Humans

• Talk to me about your data and your humans in your projects

@juggernautco

Data

• Dense• Sits by itself• Not social• Not self-aware• Unable to contextualize itself• Does not have any problems, because it

doesn’t care about anything

@juggernautco

People

• Naturally social• Soft• Have problems• See everything in context• Prone to mistakes

People make data

@juggernautco

@juggernautco

@juggernautco

@juggernautco

@juggernautco

Value from data

• Know more than anyone • Surfacing from the hidden Web• Context, context, context• Even if it is just one data set mashed against

another data set• Did it rain * Did property crime go up or down• Foreclosures * Retail stores• Also: the simple act of aggregation + text

@juggernautco

@juggernautco

Ten Databases

• Building permits• Business licenses• Historic preservation list• Sanborn maps (1929 and 1950)• County assessor • County recorder of deeds• Original photography• Google search for news coverage• New York Times archive• Walgreens surplus property

@juggernautco

We need a machine.

• A generic context engine• To evenly distribute information• And tell me what the information

means• I know: that sounds like a “reporter”• But people used to think that

“search engine” sounded a lot like “librarian”, too

• We need humans and machines

@juggernautco

It’s easy.

• Find dataset• Review dataset• Describe what the data means• Find another dataset• Describe what the other dataset

means• Describe what the first dataset means

in the context of the second dataset• Repeat• Let’s do this thing.

@juggernautco

Dedicated databases work

@juggernautco

Call any time.

• @juggernautco• (773) 960-6045