Upload
vincenzo-patruno
View
3.265
Download
6
Tags:
Embed Size (px)
DESCRIPTION
Presentation about the most interesting SDMX tools. How tio start working with SDMX using the ISTAT SDMX Framework, a set of building blocks to easily set up an SDMX architecture
Citation preview
SDMX TOOLSWorkshop on Data Transmission
Becici – Montenegro17- 19 June 2009
Vincenzo PATRUNOISTAT
Overview
There are several tools that help us working with SDMX files
Released under GPL or EUPL license
http://sdmx.org/?page_id=13http://sdmx.org/?page_id=13
http://www.osor.eu/communities/oss-and-statistics/wiki/sdmx-toolshttp://www.osor.eu/communities/oss-and-statistics/wiki/sdmx-tools
SDMX Converter
http://sdmx.org/?page_id=13http://sdmx.org/?page_id=13
SDMX-IM
From SDMX-IM to messages
SDMX-EDI SDMX-ML
• Based on a common Information Model– SDMX-EDI (GESMES/TS)
• EDIFACT syntax• Time series oriented – One format for Data Sets
– SDMX-ML• XML syntax• Four different formats for Data Sets• Easier validation (XML based)
From SDMX-IM to messages
SDMX-ML
XML format for the exchange of SDMX-structured data and metadata.
Name of message Short description Schema file
1 Structure Definition Message Contains a data structure definition Fixed
2 Generic Data MessageConveys data in a form independent of a data structure definition. It is designed for data provision on websites and in any scenario where applications receiving the data may not have detailed understanding of the data set's structure before they obtain the data set itself.
Fixed
3 Compact Data Message Exchange of large data sets in a data structure definition-dependent form
Derived from data structure definition message
4 Utility Data Message For schema-based functions, such as validation, in a data structure definition-dependent form
Derived from data structure definition message
5 Cross-sectional Data Message
Exchange of many observation types in a data structure definition-dependent form
Derived from data structure definition message
6 Query message To query a database to obtain an SDMX-ML message as the result Fixed
SDMX-ML: Six standard messages
Metadata
Structural metadata must be associated with the data, otherwise it becomes impossible to identify, retrieve and browse the data.
The simplest explanation of a data structure definition is that it is a set of descriptor concepts, associated with a set of data, which allow us to understand what that data means.
Data Structure Definition
SDMX-ML “Model-Driven” XML Approach
SDMX Converter Main Activities
Reading the input message parsing of the message populating the data model of the tool (based on the SDMX v2.0
information model ) Writing the converted message
uses the data model to write the output message in the target format.
Supported formats SDMX-ML (Generic, Compact, Utility Compact) Gesmes (TS, 2.1. DSIS) Flat files (CSV, FLR)
DSD retrieved from the Registry that is mandatory for completing a conversion
http://bms.istat.it/sodidownload/download.aspxhttp://bms.istat.it/sodidownload/download.aspx
ISTAT SDMX Framework
SDMX Framework It is based on the SDMX Information Model It covers all the dissemination business process It can be used entirely
or Only some building bocks can be used, in order to integrate them in an already existing information system
Available APIs SDMX Query Parser
It allows to set up an SDMX architecture in a short time It can be used for didactic purpose It will be available as Open Source
Technical requirements windows applications:
MS Windows XP or sup. Microsoft .Net framework 2.0
web applications MS Windows 2003 or sup. Internet Information Server 6.0 or sup. Microsoft .Net framework 2.0 Microsoft ASP.NET 2.0
Database Oracle 9 or sup MS SQL Server 2005 or sup. MS Access
NSI
1
2
3
Data Sharing Model
Query SDMX
DataSDMX-
ML
RSS
WSNSI
Data Sharing Model
Link to SDMX
file
DataSDMX-
ML
RSS
WSNSI
Data Sharing Model
Query Message
This is another special-purpose message. It is used to convey a query to a database which then returns an SDMX-ML message
For web services and database-driven applications
Data Structure Wizard
http://sdmx.org/?page_id=13http://sdmx.org/?page_id=13
SDMX Registry A repository for keeping
Structural metadata (e.g. CodeLists, ConceptSchemes, DSDs, MSDs)
Provisioning information (e.g. Dataflows, Provision Agreements)
Repository is accessible via a Web Service accepting SDMX-ML messages
GUI for user interaction over the Web Authentication using CIRCA accounts DSW – “standalone” Java GUI
https://webgate.ec.europa.eu/sdmxregistry/
SDMX Registry