( It used to be called “transparency,” but then things changed. )

The Global

Open Data Movement

Tom JohnsonInstitute for Analytic Journalism

Santa Fe, NM USAt o m @ j t j o h n s o n . c o m

1

2

Presentation for the Men’s Lunch Bunch

Christ Evangelical LutheranSanta Fe, New Mexico

Feb. 17 2012

PowerPoint slides at TK

The Global Open Data Movement by J. T. Johnson is licensed under a

Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License

.

How did this find me?

3

• Doing workshop in UK• International students • Same week, call for action in

Poland, Mexico and Guatemala• Started following #opendata and

listservs• Global interest – and action --

quickly apparent

What is the Open Data Movement

4

• “The People” taking control of their public data

• Global awareness of digital data• Gov’t, NGOs, corporate, research

institutions, “people postings”• Recognized value of collaboration

• Rarely a single person task• DB experts, code jockeys, UI specialists,

journalists, project managers

History of Open Data

5

• The Visigoths: perhaps the earliest open-law law

• Flour inspectors in the 1760s, and codification of law

• Disbursements disclosure in 18th century China

• The first FOI law in 18th century Sweden

• Today, >90 nations have FOIA laws; increasingly turning to Open Data

History of Open Data

6

• The Visigoths: perhaps the earliest open-law law

• Flour inspectors in the 1760s; codification of law

• Disbursements disclosure in 18th century China

• The first FOI law in 18th century Sweden

• Today, ~90 nations have FOIA laws; increasingly turning to Open Data

Data Sources

7

• Gov’t at all levels, incl. international• NGOs• Non-profits• Corporate

Date types• ~400-500 different file types/formats

OD – trending to critical mass

8

Cheap, ubiquitous hardware/software

9

10

Asahi Soft Drinks offers free Wi-Fi in vending machines

“The drinks company is to build into each of its vending machines a Wi-Fi router which will offer connection to Japanese network Freemobile.

“Those in the immediate vicinity can then hook up for free high-speed Internet access on their mobile devices or laptops for up to 30 min-utes in one sitting. The router will also store localised content, such as shopping mall maps, regional infor-mation, tourist details, that sort of thing.

“Oh, and they will sell soft drinks, of course.”

OD – trending to critical mass

11

Cheap, ubiquitous hardware/software

Ubiquitous digital skills

12

OD – trending to critical mass

13

Cheap, ubiquitous hardware/software

Ubiquitous digital skills

Affordable, accessible, significant bandwidth

June 2007: The price of Internet access is under a dollar per month for each megabit per second in just three countries -- Japan, Korea and Sweden. The cost in all of the top 10 countries averages less than $5 per month/megabit.

Achieve critical mass c2011

14

Cheap, ubiquitous hardware/software

Ubiquitous digital skills

Affordable, accessible, significant bandwidth

Global Open Data Movement

15

Global Open Data

16

2011/12/8 Keserű Júlia <keseru.julia@k-monitor.hu>:> Dear Marco,>> K-Monitor Watchdog for Public Funds from Hungary has been working on a> database (called the Network) which scrapes open government data from the> following areas: public procurements, EU tenders and agricultural subsidies> as well as company registry and personal information of politicians and> business people who are close to public funds. Later on, we'll try to add> legislation data to the Network and though we have no public information on> party or campaign financing, one our major goals was to make a database for> detecting conflicts of interest.>> The database is in a beta version yet and will be launched in January, if> you need any more information feel free to contact me!>> Bests,> Julia Keserû> Budapest, Hungary

World Map of Open Government Data Initiatives

17

Public Dataset Catalogs Faceted Browser137 Catalogues of public data

18

Chicago Shovels

19

Human Sensor Networks

20

How does it work?

21

• Collaboration (and tools for) key• British MP expenses

• Volunteer hack-athons• Smart-phone apps

• Local gov’t – NYC tree census• Small-shop code jockeys

22

• Fine-grained data, NOT just summaries• School test scores• Casino revenues tax revenues• Local elections “chain of custody,”

reported voting results and “audits”• Public transit ridership & costs• Actual language of all gov’t contracts

• Access to performance measurement metrics

• No PDFs

New pressures on gov’t , all institutions

Tectonic change

23

• The document per se isn’t important, the data is

• Data only valuable with analysis, i.e. “making meaning”

• The data – and/or related reports – only valuable if its genealogy is known

• i.e. Show us the methodology

24

• Require methodology documentation with data/reports• Increase skill set and cost of government• ROI

• Less duplication of data collection and analysis

• New tools for citizens• Identify potential for business

opportunities

New pressures on gov’t , all institutions

Inherent contradictions

Challenge of “Big Data/Open

Data”

Desire to retain institutional and

individual “power” via data

25

• Exponential growth

• Sorting important from interesting

• Understanding significance

• Two sets of books• Hiding or skewing

data• Fear of making

mistakes

The push-back

26

• Already signs of “disappearing” data• Medical doctors reports taken down

The push-back

27

• Already signs of “disappearing” data• Medical doctors reports taken down• Ash emission from coal-burning plants• Difficulty – sometimes because of lack

of reporting standards – in getting data related to petroleum industry

Tracking the topic

28

• Resources• The Open Knowledge Foundation Wiki• Open Data Cookbook • International Open Government Dataset

Catalog (IOGDC)• Open Data Showroom

• Collect related Tweets with #opendata

• Various listservs (language specific)

( It used to be called “transparency” )

The Global

Open Data Movement

Tom JohnsonInstitute for Analytic Journalism

Santa Fe, NM USAt o m @ j t j o h n s o n . c o m

29