Upload
nguyendung
View
217
Download
0
Embed Size (px)
Citation preview
«
Technical coordination
Work packages
Dissemination (wiki, web site)
«
«
ESSnet on SDMX – Technical issues
«
«
ESSnet on SDMX – Technical issues
Objectives
The main contents of the terms of reference:
• Implementation of SDMX at international and
national level
• The further development of the SDMX standards
used for data and metadata
•The further development of the SDMX Content-
Oriented Guidelines
• The further development of the SDMX IT architecture
and IT tools
• The increase of SDMX capacity and knowledge
«
«
ESSnet on SDMX – Technical issues
Objectives
Broadening and strengthening a NSI standard SDMX framework
(WP8);
Making standard tools available to a more easy implementation of
the standard SDMX in the processing and exchange of data and
metadata files between national and international organizations (WP3,
WP4, WP7);
Improving the semantic harmonization of concepts and terminology
supporting this standard (WP2);
Disseminating related tools and documentation widely (WP9);
«
«
ESSnet on SDMX – Technical issues
Objectives
• Results have to be broadly reusable within the ESS
• Software tools developed on an Open Source basis and registered
under European Union Public licence (EUPL);
Ensures the following rights:
Obtain the source code of the software from a free access repository;
Use the software in any circumstances and for all usage
Reproduce (copy, duplicate) the software
Modify the original software
The development of software should take care about:
A modular design
The use of a version control system
Clear documentation
• Detailed information at http://ec.europa.eu/idabc/eupl
«
«
ESSnet on SDMX – Technical issues
Success factors
Communication: all the members of the project shall communicate
anything that might affect or contribute to the project;
Good inter co-ordination between Work Packages, namely
between WP’s responsibles , in order to minimize the overlapping of
work and to assure interoperability as always as needed;
Strong involvement of all the participants and stakeholders;
Allocation of the right people for each task ;
Stick to the approved working plan, reporting as soon as possible
any problem that could compromise it;
«
«
ESSnet on SDMX – Technical issues
Main achivements
• Involvement and increase of knowledge on NSI’s about SDMX;
• Better inter co-ordination between Work Packages;
• Improvement of ROI ( Return On Information);
• A common framework to support integrated and interoperable
solutions;
• Dramatic reduction of developments costs on each NSI, due to
common tools and solutions;
«
«
ESSnet on SDMX – Technical issues
Technical coordination
WP1 – Management of the Action (PT)
Work:
• Monitoring the work in the different WP’s, the coordination between WP’s and the quality of the deliverables;
• To Identify problems and make proposals to overcome them;
• To Produce progress reports;
• Organizing coordination meetings;
Meetings:
• Kick-off meeting – 16/10/2009 - Eurostat and partner countries:Start the project, make a project review, establish theproject governance, define roles and responsibilities, clarifythe expectations of all parties;
• 2 nd coordination meeting – 26/03/2010 - progress evaluation
Reports:
Quarterly progress reports (January, April)
Technical coordination
Difficulties:
Sound knowledge of the state-of-the-art is needed
In SDMX field there are too many software tools not yet
broadly known and commonly accepted
There are other standards for the same purpose
Oportunities:
Increase knowledge on SDMX
An oportunity to implement SDMX in the Organization
The possibility of influencing the way forward on the short and
medium run
«
«
ESSnet on SDMX – Technical issues
Proposals
A group of experts to assist the coordinator on
technical evaluation of WP’s would be desirable;
A repository of specialized tech skills existing on
each NSI, could resolve eventual gaps on a particular
task, i.e. sharing knowledge not just information;
Improvement on the current practise of reusing
existing pieces of software already developed by
someone in any project.
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP2 – MCV Ontology (PT)
Responsible: Statistics Portugal
Goal: To create an ontology of statistical metadata based
on Metadata Common Vocabulary (MCV) concepts
Deliverables: 1st version of the ontology - July 2010
2nd version of the ontology – November 2010
Final version of the ontology – March 2011
Report describing the methodology used – March 2011
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP2 – MCV Ontology (PT)
Terminological point of view
Improve the quality of the definitions:
More complete and accurate
More clear
More coherent among them
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP2 – MCV Ontology (PT)
Technological point of view
Semantic Web Technologies(*)
Web services – software tools designed to support interoperable machine-to-machine
interaction over a network
Ontology – Set of concepts, the relations among them and software
for computing them
Knowledge representation languages – RDF, OWL (Web Ontology
Language)
Turn an ontology from a model of concepts into something that is
computational
Make inferences with the concepts in an ontology
(*) The ideas and definitions in this slide were extracted from “Open issues on the semantic web”, Dan Gillman,
Work session on statistical metadata (METIS), March 2010
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP3 – SDMX Data Explorer (PT, IT, ES)
Responsible : Statistics Portugal
Participants: ISTAT, INE-SP
Goal: To deliver a tool (using a generic SDMX class library) to
visualize SDMX data files under different perspectives
(html, CSV, Charts, SDMX format conversion) in a more
user friendly way.
Progress: Functional and technical specifications (03.D-1 finished);
Analysis and design documentation (03.D-2 finished);
User Interface prototype description(03.D-3 finished);
The development of SDMX-ML Parser has been started;
Source code available for partners in repository
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP3 – SDMX Data Explorer (PT, IT, ES)
Deliverables: Application package intended to explore statistical data
using an SDMX data file and related DSD;
Documentation associated with the tool:
Functional and technical specifications
Analysis and Design documentation
User interface prototype
User Manual
Implementation report
Application package: Web application, class
library and web services
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP4 – Generalized SDMX Framework (IT, PT, SK)
Responsible: ISTAT
Participants: Statistics Portugal, INFOSTAT (Slovak Republic)
Goal: To complete the “Generalized SDMX Framework” already
implemented by ISTAT, with more functionalities, in order
to make it available as a complete product;
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP7 – PC-Axis with SDMX-ML (SE, NO, IT)
Responsible: SCB (Sweden)
Participants: ISTAT, Statistics Norway
Goal: To create a PCAxis <–> SDMX converter, based on
PCAxis files and on an SQL database structured
according to the Nordic Metadata Model
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP8 – SDMX in Survey Design and Data Collection (SK, PT)
Responsible: INFOSTAT (Slovak Republic)
Participants: Statistics Portugal
Goal: To implement SDMX in the survey design and data
collection
Pilot implementation on a selected statistical
domain (tourism and census);
Progress: Design of system architecture and system functionality
(May 2010) – 1st deliverable
Deliverables: Pilot implementation results
System documentation
Report on the methodological results achieved
«ESSnet on SDMX – Technical issues
Work Packages
Implementation of SDMX in survey design and data collection
processes at NSI
METADATA
MDSD
FORMS
SURVEYSDATA
STRUCTURE
DEFINITIONS
SDMX DB
RESPONDENTS
REGISTRY
SOURCE
DB
eFORMS
REPOSITORY
e FORMS
GENERATOR
DOCUMENTS
GENERATOR
RESPONDENTS
RESPONSES
MANAGER
LOADER
METADATA
MANAGER
EXTERNAL
METADATA
SOURCE DATA DATA
EDITING
IMPUTATION
REGISTER
DATA
LOADER
System
architecture
«WP8 – SDMX in survey design and data collection
«Proposal for e-form (e-questionnaire) standard
Preamble section
Identification section (zero modul)
Information section
Interpretative note section
Data section (modules)
Declaration section
«WP8 – SDMX in survey design and data collection
«Structure of preamble section
LOGO Official name of the statistical office
Statistical survey nameForm ID
Surveyed period
Declaration of data protection
Form registration No.Address of form submitting
Date of submitting
Subbmitt to:
Until:
Contact::
Contact address
«WP8 – SDMX in survey design and data collection
«
Matrix Data Strucure
Heading of columnsHeading of rows
Numbering
of rows
Numbering
of columns
Variable
Matrix
«WP8 – SDMX in survey design and data collection
«Report structure for describing e-form
The structure of report on concepts in Concept Scheme for filling the tables describing statistical e-form - Heading level and Modules level (descriptive metadata for database tables) Relationships Concepts Sub-concepts The new semantic role of the
concept, which is used from another concept
1. .1 ORGANISATION_RESPOSIBLE The organization responsible for survey
CONTACT_ORGANISATION 12.1 Responsible organization
Abbreviation
LOGO
1. .1 FORM_IDENTIFICATION Identification Form
CONTACT_NAME 12.3 Statement founded
SURVEY_TITLE
SURVEY_CODE
IKF_CODE
SURVEY_YEAR
SURVEY_PERIOD
REGISTRATION_PLACE
REGISTRATION_NUMBER
REGISTRATION_DATE
1.. N IDENTIFICATION_KEY Identification Items
ROW_IDE
KEY_ORDER
KEY_NAME
KEY_SIZE
INSTRUCTFILL
1.. N INDICATION_DATA Indicator data
ROW_IND
KEY_ORDER
KEY_NAME
KEY_SIZE
INSTRUCTFILL
1.. N KCOMPLEMENTARY Descriptive information on the identifier and indicator
COMPLEMENTARY
COMPL_ORDER
1. .1 CONFIDENTIALITY Confidentiality
CONF_POLICY
CONF_DATA_TR
1. .1 INST_MANDATE Institutional mandate
Addressing
INST_MAN_LA_OA
INST_MAN_SHAR
1. .1 DELIVER_CONTACT Data for service form
DELIVER Deliver - description
CONTACT_NAME 12.3 Deliver - the person's name
CONTACT_ORGANISATION 12.1 Deliver - Organization (Regional Report)
CONTACT_MAIL 12.5 Deliver - postal address
1:1 METHODOLOGICAL_ EXPLANATION
Methodological notes on title
EXPLANATION_HEAD
EXPLANATION
1.. N SURVEY_MODULES
MODULE_ORDER
MODULE_SIGN
MODULE_NAME
The structure of report on concepts in Concept Scheme for filling the tables describing statistical e-form - Heading level and Modules level (descriptive metadata for database tables) Relationships Concepts Sub-concepts The new semantic role of the
concept, which is used from another concept
1. .1 ORGANISATION_RESPOSIBLE The organization responsible for survey
CONTACT_ORGANISATION 12.1 Responsible organization
Abbreviation
LOGO
1. .1 FORM_IDENTIFICATION Identification Form
CONTACT_NAME 12.3 Statement founded
SURVEY_TITLE
SURVEY_CODE
IKF_CODE
SURVEY_YEAR
SURVEY_PERIOD
REGISTRATION_PLACE
REGISTRATION_NUMBER
REGISTRATION_DATE
1.. N IDENTIFICATION_KEY Identification Items
ROW_IDE
KEY_ORDER
KEY_NAME
KEY_SIZE
INSTRUCTFILL
1.. N INDICATION_DATA Indicator data
ROW_IND
KEY_ORDER
KEY_NAME
KEY_SIZE
INSTRUCTFILL
1.. N KCOMPLEMENTARY Descriptive information on the identifier and indicator
COMPLEMENTARY
COMPL_ORDER
1. .1 CONFIDENTIALITY Confidentiality
CONF_POLICY
CONF_DATA_TR
1. .1 INST_MANDATE Institutional mandate
Addressing
INST_MAN_LA_OA
INST_MAN_SHAR
1. .1 DELIVER_CONTACT Data for service form
DELIVER Deliver - description
CONTACT_NAME 12.3 Deliver - the person's name
CONTACT_ORGANISATION 12.1 Deliver - Organization (Regional Report)
CONTACT_MAIL 12.5 Deliver - postal address
1:1 METHODOLOGICAL_ EXPLANATION
Methodological notes on title
EXPLANATION_HEAD
EXPLANATION
1.. N SURVEY_MODULES
MODULE_ORDER
MODULE_SIGN
MODULE_NAME
…
«
«
ESSnet on SDMX – Technical issues
Work Packages
WP9 – Dissemination (PT, all co-partners)
Responsible: Statistics Portugal
Participants: All partner countries
Goal: Dissemination of the results of this Action within
and outside the European Statistical System
Progress: ESSnet on SDMX web site (available April 2010)
http://sdmxessnet.ine.pt
Deliverables: Training Plan on the tools produced
Presentations about the project (ITDG, STNE,
MWG)
Final workshop (March 2011, Lisbon)
«
«
ESSnet on SDMX – Technical issues
Dissemination
WIKI and WEB SITE
WIKI
Populated by the participants on the ESSnet
Authentication needed
Documents can be uploaded on draft status
Collaboration
WEB SITE
Populated by the co-ordination
Public access
Documents available with on final status
No collaboration
Structure of the wiki content at 19/03/2010
ESSnet
Main page
ESSnet on SDMX
Coordinator
Project details
Project description
Main objectives
Main milestones
Work Packages
Participants
«
«
WP
Main page
WP – name of the WP
(what is expected):
Participants
Main objectives
Description of the work
Deliverables
(In progress):
Structure of the wiki content at 19/03/2010