Embed Size (px)
Citation preview
COMBINING DATA THROUGH STANDARDS AND METRICS
Mike Thacker – Porism Limited
Open Data
Combining data through standards and metrics
2
WHAT WE MEAN BY STANDARDS
Combining data through standards and metrics 3
Making local data useful
• Municipalities need to be compare with one another
• Innovators need to build apps cost-effectively
Hence we offerstandards for:
• what we call things
• format of open data
Defining datasets and linking via URIs
Datasets
Schemas
Data
items
Def
ine
stru
ctu
re o
f
Co
nta
in
Local authorities
Official geographies
Neighbour-hoods
Services grouped by function
Other eg:
• Planning categories
• Entertainment types
UNIFORM RESOURCE IDENTIFIERS (URIS)
Combining data through standards and metrics 6
URIs - Uniform Resource Identifiers
Codes that give precise definitions of things, eg:
• http://id.esd.org.uk/service/860 – premises licence service
• http://opendatacommunities.org/id/unitary-authority/yorkYork City Council
• http://statistics.data.gov.uk/id/statistical-geography/E06000014 York area - from ONS
These normally resolve to descriptions with properties that are human and machine readable
Finding a URI - uris.opendata.esd.org.uk
What we know about each service
Service
Web page for each English council
URIs resolved - service
standards.esd.org.uk
STANDARDS FOR DATA FORMATS - SCHEMAS
Combining data through standards and metrics 11
The inventory schema
• Indexes datasets & their schemas against functions & services
• Automatically harvested by data.gov.uk
• Automatically output by DataShare DKAN and CKAN following
• Can be uploaded to and validated by esd-toolkit
Inventory
Dataset Documents
ODF, PDF, HTML
Data
CSV, XML, ...
Dataset Documents
ODF, PDF , HTML
Data
CSV, XML, ...
inventory.esd.org.uk
Dataset schemas
• Define the structure of data for a service or function (group of services)
• Shared to allow (not mandate) consistency
• Some validated schemas encouraged
• Formats:
– Tabular: DataShare definitions and CSV validation files as used by the ODI’s csvlint.io
– XML
– Linked data profiles
CSV Checker - http://csvchecker.opendata.esd.org.uk/
presentatienaam 14
Inventories of datasets
Combined index of local datasets
Datasets from many municipalities using the same schema
One feed of all aggregated CSV datasets
METRICS – STANDARD MEASURES
Combining data through standards and metrics 18
Policy Policy Metrics Services
Increase healthiness / quality of life
ObesityPsychiatric illnessCardiac illness
Dietary advice, School mealsGreen spaces,Recreational facilities
Increase economic activity EmploymentStreet crimeEducational attainment
Careers advicePolicing, CCTVSchooling, Adult education
Safer roads Road accidents Traffic control, Signage
Metrics
Policies Servicesdetermine
Evidence-led policy
Metrics structure
Metric value
Date/Time
Dimensions Area
Organisation
Examples• Miles of roads• Road accidents• Spending on roads
Example• Norfolk County
Council
Example• Norfolk administrative area• East of England• England
At a point or
Over a period
Examples• Age• Gender• Severity
Metric type
2012 6,142.5
183,003.4
2010-12
353
2,47324,027.8
21,534
£75.3M
EoE £573.2M England £208.8M
2011/12
has
has has has
has
gove
rns
Example data analysis – Comparing Municipalities
Full report: Live PDF
Further information
• UK local government open data: http://opendata.esd.org.uk/
• Standards: http://standards.esd.org.uk/
• Metric types http://id.esd.org.uk/list/metricTypes
• Dimensions (Circumstances) http://id.esd.org.uk/list/circumstances
• Municipalities http://opendatacommunities.org/data/local-authorities
• Administrative areas http://statistics.data.gov.uk/doc/statistical-geography/
• Natural neighbourhoods: http://neighbourhoods.esd.org.uk/
• Tools http://about.esd.org.uk/
• API: http://api.esd.org.uk/
@MikeThacker
