Lse open data construction and critiques

Version 1 - 21st February 2012 - LSE Digital Media Futures Course Guest Lecture [email protected] | @timdavies | http://www.timdavies.org.uk

Open government data: construction and critiques...

Open data: construction and critiques

mailto:[email protected]


http://www.timdavies.org.uk




Part 1: 5 perspectives on open data

Part 2: Unpacking open government data

Part 3: How did we get to here?

Part 4: Claims and critiques

Endnotes: Where next?







Part 1:5 perspectives on open data







1. Open data is a set of policiesand practices







Accessible

Standardised

Re-usable

Place data online

Use common formats

Use open licenses

.csv, .xml, .txt

Open data should be...







2. Open data is a response to changingtechnologies, and societies







Technical change: Bandwidth and processing capacity to work with data has grown. We can share all layers, not just analysed information. This creates conditions, and demand, for open data.

Social change: As authority becomes decoupled from institutions, more people outside the establishment want to check on the basis of ‘evidence based practice’. We want to create our own representations, not just accept those provided by others.

Diagram

from ‘O

pen Data, D

emocracy and Public Sector R

eform’, T

im D

avies, 2010 - http://practicalparticipation.co.uk/odi/report/2010/2-3-data-and-inform

ation/#3





http://practicalparticipation.co.uk/odi/report/2010/2-3-data-and-information/#3

http://practicalparticipation.co.uk/odi/report/2010/2-3-data-and-information/#3



3. Open data is what open data does







Map of Esmee Fairbairn Grantmaking based on scraped data and OpenCharities.org datasets; Live Tube Map from http://traintimes.org.uk/map/tube/Bubble chart from http://www.guardian.co.uk/uk/datablog/interactive/2011/dec/06/england-riots-crimes-arrested

Most people’s direct experience (and idea) of open

data is of mash-ups, visualisations, websites and

mobile apps





http://traintimes.org.uk/map/tube/

http://traintimes.org.uk/map/tube/

http://www.guardian.co.uk/uk/datablog/interactive/2011/dec/06/england-riots-crimes-arrested

http://www.guardian.co.uk/uk/datablog/interactive/2011/dec/06/england-riots-crimes-arrested



4. Open data is a philosophy& a movement







On the one hand, recent decades have seen an elaboration of ideas of Intellectual Property, and an emphasis on information as a valuable

commercial asset: owned and enclosed by governments or corporations. On the other, open source ideas and a reaction to this new era of enclosure have inspired revitalised movements for access to knowledge (A2K) and an information

commons. Openness is seen as both an ethical good, and the only way to support global collaboration on the issues that matter.







5. Open data is a foundationfor a collaborative web







World Wide Web leader, Sir Tim Berners-Lee, envisaged a web not just of documents, but also of datasets - allowing people to collaborate with less friction

across organisational boundaries.

A web of open data is the next step of ‘weaving the web’.

























Recap: 5 perspectives on open data

1. Open data is a set of policies and practices

2. Open data is a response to changing technologies, and societies

3. Open data is what open data does

4. Open data is a philosophy and a movement

5. Open data is a foundation for a collaborative web







Part 2:Unpacking open government data







Open government data

• We make decisions based on information

• Information = data + context

• Data = encoded information

• Data can be manipulated by computers: sorted, filtered, remixed and turned back into information in diverse ways








Data as a representation of the world...
































There are many contemporary narratives about ‘data’...









Raw data









Raw dataBig data









Raw dataBig data

MiData









Raw dataBig data

MiDataLinked data









Raw dataBig data

MiDataLinked data

Open data









Raw dataBig data

MiDataLinked data

Open data

...








• Governments have historically taken on a monopoly role as data collectors

• Government data collection shapes as well as records the world

• The government data that is available is the product of political decisions

• Is a government monopoly on data collection still necessary?















Access; Redistribution; Re-use; No DRM; Attribution; Integrity;

Non-discrimination...

http://opendefinition.org/okd/










http://web.idrc.ca/uploads/user-S/12271304441Open_ICT4D_Draft.pdf

Access; Redistribution; Re-use; No DRM; Attribution; Integrity;

Non-discrimination...












Part 3:How did we get to here?







J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM2003

2004

2005

2006

2007

2008

2009 2010

April 2003Advisory Panel on Public Sector Information (APPSI) is established to advise on and encourage “opportunities in the information industry for greater re-use of public sector information”.

November 2003European Public Sector Information Directive is adopted. States must implement it by July 2005.

January 2009President Obama issues Memo on Transparency and Open Government as one of his first acts in Office.

July 2008Office of Public Sector Information Data Unlocking Service is launched.

July 2005Re-use of Public Sector Information directive comes into force.

January 2005Freedom of Information Act comes fully into force.

June 2004TheyWorkForYou.com is launched at NotCon '04 using data on political decision making scraped from the Hansard website. The site is later adopted by MySociety.org.

May 2008“Models of Public Sector Information Provision via Trading Funds” (Newbury, Pollock et. al 2008) published alongside the 2008 UK Budget which included commitment to ensure information from Trading Funds “is made available as widely as possible for use in downstream markets”

March 2008Interim Power of Information report published and Power of Information taskforce established

October 2007The District of Columbia launch the first major government data catalogue bringing together data formerly published in different areas of the DC website, and creating pressure for the release of further data.

January 2007Power of Information Report commissioned. Co-authored by MySociety director Tom Steinberg and Ed Mayo.June 2006

MEPSIR (Measuring EU Public Sector Information Resources) Study estimates mean potential value from PSI re-use across Europe at €27bn.

June 2008Show Us a Better Way competition announced.

October 2008Apps for Democracy programme run to encourage re-use of data from the DC data catalogue.

September 2009The Cabinet Office Director of Digital Engagement issues a call for developers to help test the new Data.gov.uk site launched in developers beta. The launch includes a number of datasets in RDF.

June 2009Tim Berners-Lee and Nigel Shadbolt appointed as advisors to the UK Government on opening up access to government data.

May 2009Data.gov launched in the US with 47 datasets.

March 2009First Rewired State Hack the Government Day held in the UK, with 80 developers building applications - many of which involved scraping government data.

February 2009Final Power of Information Report published.

February 2009Tim Berners-Lee talks to the TED Conference about the need for Raw Data Now from governments.

March 2009Raw Data on Cycle Accidents released through the Directgov Innovate programme. A range of mash-ups are created, and the cycle data is frequently cited as an example of the potential of releasing raw data.

March 2010Further Rewired State events take place, with backing and funding from UK Government departments.

January 2010Data.gov.uk officially launched and made availble to all. London also launches datastore for the capital.

December 2009Putting the Frontline First strategy commits government to greater openness in the release of data. The report also establishes a Local Data Panel to focus in the release of local authority data.

December 2009The US Open Government Directive establishes deadlines for US Government Departments to get more information online.

April 2010Ordnance Survey release significant GeoData as open data following a long 'Free our data' campaign by the Guardian, and consistent calls from developers for better access to geodata.

May 2010Prime Minister David Cameron sends a letter to government departments asking for specific action on opening up government data and establishes a new 'Transparency Board' to include Tim Berners-Lee, Nigel Shadbolt, Tom Steinberg (Founder of MySociety) & Rufus Pollock

June 2010COINS (UK Government Spending) dataset released.

#opendata TimelineFocussing on UK Open Government Data

Draft 0.1 by @timdavies for Open Data Impactsresearch project: http://www.practicalparticipation.co.uk/odi/

Please credit and link to the URL above. Direct any enquiries to [email protected] and add to the underlying data at http://bit.ly/ak4gYx

Government Led

Community Led

Research

UK

US

Europe

Milestone Event Other Events

Key

March 2006Guardian 'Free Our Data' campaign is launched.







J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM J F M A J J A OS N DM2003

2004

2005

2006

2007

2008

2009 2010

April 2003Advisory Panel on Public Sector Information (APPSI) is established to advise on and encourage “opportunities in the information industry for greater re-use of public sector information”.

November 2003European Public Sector Information Directive is adopted. States must implement it by July 2005.

January 2009President Obama issues Memo on Transparency and Open Government as one of his first acts in Office.

July 2008Office of Public Sector Information Data Unlocking Service is launched.

July 2005Re-use of Public Sector Information directive comes into force.

January 2005Freedom of Information Act comes fully into force.

June 2004TheyWorkForYou.com is launched at NotCon '04 using data on political decision making scraped from the Hansard website. The site is later adopted by MySociety.org.

May 2008“Models of Public Sector Information Provision via Trading Funds” (Newbury, Pollock et. al 2008) published alongside the 2008 UK Budget which included commitment to ensure information from Trading Funds “is made available as widely as possible for use in downstream markets”

March 2008Interim Power of Information report published and Power of Information taskforce established

October 2007The District of Columbia launch the first major government data catalogue bringing together data formerly published in different areas of the DC website, and creating pressure for the release of further data.

January 2007Power of Information Report commissioned. Co-authored by MySociety director Tom Steinberg and Ed Mayo.June 2006

MEPSIR (Measuring EU Public Sector Information Resources) Study estimates mean potential value from PSI re-use across Europe at €27bn.

June 2008Show Us a Better Way competition announced.

October 2008Apps for Democracy programme run to encourage re-use of data from the DC data catalogue.

September 2009The Cabinet Office Director of Digital Engagement issues a call for developers to help test the new Data.gov.uk site launched in developers beta. The launch includes a number of datasets in RDF.

June 2009Tim Berners-Lee and Nigel Shadbolt appointed as advisors to the UK Government on opening up access to government data.

May 2009Data.gov launched in the US with 47 datasets.

March 2009First Rewired State Hack the Government Day held in the UK, with 80 developers building applications - many of which involved scraping government data.

February 2009Final Power of Information Report published.

February 2009Tim Berners-Lee talks to the TED Conference about the need for Raw Data Now from governments.

March 2009Raw Data on Cycle Accidents released through the Directgov Innovate programme. A range of mash-ups are created, and the cycle data is frequently cited as an example of the potential of releasing raw data.

March 2010Further Rewired State events take place, with backing and funding from UK Government departments.

January 2010Data.gov.uk officially launched and made availble to all. London also launches datastore for the capital.

December 2009Putting the Frontline First strategy commits government to greater openness in the release of data. The report also establishes a Local Data Panel to focus in the release of local authority data.

December 2009The US Open Government Directive establishes deadlines for US Government Departments to get more information online.

April 2010Ordnance Survey release significant GeoData as open data following a long 'Free our data' campaign by the Guardian, and consistent calls from developers for better access to geodata.

May 2010Prime Minister David Cameron sends a letter to government departments asking for specific action on opening up government data and establishes a new 'Transparency Board' to include Tim Berners-Lee, Nigel Shadbolt, Tom Steinberg (Founder of MySociety) & Rufus Pollock

June 2010COINS (UK Government Spending) dataset released.

#opendata TimelineFocussing on UK Open Government Data

Draft 0.1 by @timdavies for Open Data Impactsresearch project: http://www.practicalparticipation.co.uk/odi/

Please credit and link to the URL above. Direct any enquiries to [email protected] and add to the underlying data at http://bit.ly/ak4gYx

Government Led

Community Led

Research

UK

US

Europe

Milestone Event Other Events

Key

March 2006Guardian 'Free Our Data' campaign is launched.







July 2011:Open Public Services

white paper

July 2011:opendata.go.ke

August 2010: Web foundation launch

OGD programme

April 2011:Data.gov faces funding

cuts

Continued UK Gov transparency

drive

September 2011:Open Government Partnership

launch

Proliferationof open data initiatives

across the world













Open (government) data movements

• The Public Sector Information (PSI) lobby

• Access to Information / Freedom of Information (FOI)

• Civic hacker / digitising government / computerisation movement







Part 4:Claims and critiques







“Open data will revitalise democratic engagement.”

Quote from Eric Pickles, quoted at http://www.edemocracyblog.com/political-blog/accountability-open-data-and-democracy/

Eric Pickles, Communities Secretary





http://www.edemocracyblog.com/political-blog/accountability-open-data-and-democracy/


























“Secondly, one of the common justifications for opening government data is that it obviates the need for intermediary organizations, and empowers individual citizens by providing direct access to the information government itself uses to make decisions. This basic mistrust of bureaucratic organization is reflected in the frequent claim made by technologists that governments should simply release the raw data and leave others to get on with the job of using it. Although this may be relatively successful as a model for producing socially- and economically-valuable software, this paper has suggested it has yet to prove its worth as a model for political action. “

Mcclean, T. (2011). Not with a Bang but a Whimper The Politics of Accountability and Open Data in the UK.







“Open government data will support economic growth.”







“Open government data will support economic growth.”







“Open government data will support accountability and efficiency”

Just the dataset? Raw data has to go through a lot of steps before it’s usable. Do open data initiatives have a responsibility to support those steps?







“Open government data will support accountability and efficiency”

Cleaned Data

Open Data

Source Code and

Configurations

Linkable Data

Shared Code

Repository

Service Technology

API or Cleaned Data

Dump

Mashups & Apps

Code Libraries &

Configurations

Integration

Improve Data Quality

Improve Contextual

Understanding

Constraint:imperfect

data

Complement:Geospatial or

Structured data

Complement:Analytics,

Visualisation

Complement:Open source

Complement:ExistingServices

Interlocking S1Human -> Materia

Interlocking S2Material -> Human

Interlocking S3Human -> Material

= physical artefact

= material agency

= human agency

(c) George Kuk and Tim Davies, 2011Adapted from draft of Assembling Open Data Complementarities for Service Innovation. Available at:http://www.practicalparticipation.co.uk/odi/2011/02/what-supports-the-sustainable-re-use-of-open-data/

Just the dataset? Raw data has to go through a lot of steps before it’s usable. Do open data initiatives have a responsibility to support those steps?







“...for “open data” to have a meaningful and supportive impact on the poor and marginalized, direct intervention is required to ensure that elements currently absent in the local technology and social ecosystem are in fact, made available”

Gurstein, M. B. (2011). Open data: Empowering the empowered or effective data use for everyone? First Monday, 16(2).







So: is open government data good or bad for democracy and society?







Part 1: 5 perspectives on open data

Part 2: Unpacking open government data

Part 3: How did we get to here?

Part 4: Claims and critiques


Recap















• It’s still early days for open government data. Key questions remain over the sorts of production, and reform engagement it will support in practice.

• Research is getting going - but faces a research paradox.• The openness of open data can hide the trails we might use to

trace it’s impacts

• The ‘movement’ is going global: and encountering new ideas and challenges in the process.







More at:

http://www.opendataimpacts.net

E-mail: [email protected]







