Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Dealing with Open Data at ISTAT
Giovanni A. BarbieriStatistics Italy (Istat)
The Open Data Movement
Our proposal is simple: […] the federal government […] is to provide data that is easy for others to reuse, rather than to help citizens use the data in one particular way or another
Open infrastructures that enable citizens to make their own uses of the data
Reverse the current policy, which is to regard government websites themselves as the primary vehicle for the distribution of public data, and open infrastructures for sharing the data as a laudable but secondary objective [Robinson,Yu, Zeller and Felten 2008]
2
Crowdsourcing Government Transparency
Government information that is nominally publicly available is in fact difficult to access either because it is not online or, if it is online, because it is not available in useful and flexible formats [Brito 2008]
“Structured data”
Associated structured XML file would allow a user to sort the data by ascending or descending date, alphabetically by headline or author, by number of words, and in many other ways
3
Open Data Ecosystem
“Open data are adding a new dimension to big data analytics and
giving rise to novel, data-driven innovations.”
(McKinsey Global Institute Report, Oct. 2013)
From Citizens IBM BLOG
Wide Range of Open Data Consumers
Citizens that would like to learn characteristics of the places they are, e.g.
with mobile apps showing location-specific features
Journalists that need to access data for updated and aware information
communication
Educators that are helped in theirteaching task by access to data on
different application domains
4
Official Statistics meets Open Data
Official Statistics can more easily reach such wide
range of users if conveyed through open data
Recent technologies advances in the open data
community enable new advanced dissemination
channels for Official Statistics
5
Reinforcingtrust
Getting closerto users
Reaching new users
Givinginformation
back
Improvingmetadata
Linked Open Data
Semantic Web Technological
Standards
OWL
Knowledge Representation
Linked Open Data - LOD
6
Why is Linked Data an Opportunity?
Linked Data as a semantically rich paradigm for data representation
Rich enough for the strict requirements of Official Statistics
Formal and well-defined data structures, i.e. ontologies
Linked Data as an international standard (W3C) Tools availability and independence Beyond statistical users RDF: Resource Description Framework (W3C)
(subject-predicate-object)
7
Istat’s Linked Open Data Portal - 1
Istat LOD Portal: http://datiopen.istat.itEnglish Version: http://datiopen.istat.it/index.php?language=eng
8
Platform for• Selecting• Navigating• Searching• Querying• VisualizingOpen Data
The platform allows• Direct access to data via Web Services • M2M solutions (e.g. GIS-LOD) • Data conversion• Export to productivity tools• Visualization by means of external tools
Istat’s Linked Open Data Portal - 2
9
STEP 1 STEP 3 STEP 2
Istat’s Linked Open Data Portal - 3
Steps to a «perfect» data portal
10
Istat’s Linked Open Data Portal - 4
Guided Access
Freedom of access
Typ
eo
f in
tera
ctio
n
Free AccessHu
man
b
asic
Mac
hin
e To
M
ach
ine
Navigation
Guided queries
Query REST onSPARQL EndPoint
Query via SPARQL EndPoint
Web Service
Download
Hu
man
A
dva
nce
d
11
Predefined Queries
(Set of simple and customizable queries)
Free Queries
(SPARQL Queries)
Navigation
Guided Queries
Download
Typ
eo
f in
tera
ctio
n
Hu
man
Guided Access Free Access
Interaction Modes
Freedom of access
Basic Advanced(Human) Usertechnical skills Intermediate
Free Query via SPARQL EndPoint
12
Use Case 1: Spatial Querying
App that displays on a map some population indicators of the
nearest census sections to specific GPS coordinates
LOD when accompanied by spatial information allow to access data
using spatial queries
13
Use Case 2: Federated Querying - 1
Federated query on Istat and ISPRA, i.e. the query accesses Istat and ISPRA portals
With LOD, it is very easy to compare data coming from different sources
(linked for example at territorial level)
Query on onePortal
Results dynamicallyretrieved
from both portals
14
ISPRA - The Italian National
Institute for Environmental
Protection and Research
Istat
Use Case 2: Federated Querying - 2
ISTAT data:
Census BuildingsISPRA data:
Data on land use / soil consumption
Example Query:
Municipality-level analysis of land use / soil consumption and number of
buildings by period of construction
Dynamically generated!
15
Use Case 3: Istat as Open Data Provider in SPOD
Social discussion (on the left) about a graphic representationof Census Data (on the right)
Dynamicallygenerated!
16
SPOD: Social Platform for Open Data
Conclusions
A dissemination strategy based on open data does
put the Official Statistics users at the centre:
Reaching them through different channels
e.g. apps and social media
Making easier for them to retrieve data
e.g. federated queries that make transparent
the distribution of data on different portals
Providing richer services to them
e.g. spatial querying and dynamical
visualizations
17
Open data and in particular Linked Open Data have a leading role in data innovation
for Official Statistics
• Macroscale vs microscale modeling
– Pseudo-Einstein (as simple as possible but not simpler)
– Von Neumann (agent-based modeling)
• Technological constraint enabling technology
• It widens the space of what is feasible:
– In production: our experience with SBS.Frame
– In analysis and research…
• A paradigm shift?
– Statistical mechanics vs agent-based modeling
– Just because you can doesn’t mean you should
• Back to open data
– From dissemination to release (“data liberation” at StatCan) to the
development of information
– The regulators need to introduce new rules in line with the new
scenarios (Don't think of an elephant!: know your values and frame the
debate)
One More Thing
18
Thanks to all my colleagues in Istat contributing to
the LOD Portal
Special thanks to Monica Scannapieco and
Stefano De Francisci
Questions and clarifications: contact me at
Acknowledgments and Thanks
18