View
49
Download
1
Category
Tags:
Preview:
DESCRIPTION
Essnet for SDMX Phase II WP2 PC-Axis SDMX Integration. PC-Axis International Reference Group meeting 31 May 2012 Lars Nordbäck and Ulf Sandberg Statistics Sweden. Objectives. Routine for reporting of statistics to international organisations in SDMX format - PowerPoint PPT Presentation
Citation preview
Essnet for SDMX Phase II WP2 PC-Axis SDMX
IntegrationPC-Axis International Reference Group
meeting 31 May 2012Lars Nordbäck and Ulf Sandberg
Statistics Sweden
23-04-21 1
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
Objectives • Routine for reporting of statistics to international
organisations in SDMX format• To set up routines to send the data in accordance
with the Eurostat Reference Infrastructure• Use the dissemination databases and PC-Axis
environment as sources at the NSI
February 2012:
Now restricted to cover only CensusHub project domain. No other domain at Statistics Sweden has DSD and code lists available within the duration of this project. No encryption facility like in eWA portal available in the RI.
23-04-21 2
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
23-04-21 3
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
Implementation Status of Domains
Balance of Payments
National Accounts
Research & Development
Pesticides
Waste Census Aviation
Fisheries
Local Coordinator meeting October 2011
Activities Only for CensusHub
• Continue the work made in Phase I, for example test the full routines coordinated with Eurostat.
• Manuals for users and administrators at the NSI in the PC-Axis community
• More use cases added to the users manual for the Process, deleted
• Continue the verifying of new versions of the Reference Infrastructure
23-04-21 4
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
Links to visualisation tools based on Generic SDMX from PC-Axis
23-04-21 5
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
1. Data Express working
2. eXplorer test is going on, with some resulting bug fixing in the conversion software
Deliverables Only for CensusHub
• Documentation. Documents to be delivered is use cases with different type of materials (2012).
• Workshops. The purpose with the workshops is to transfer the methods and software for use of the same platform based on PC-axis and the Eurostat SDMX Reference Infrastructure achievements in Phase 1 and Phase 2
23-04-21 6
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
Deliverables (cont.)
• SDMX functionality integrated in the web-software• Implementing of attributes on cell level in PX-Web• Implementation of the Generic Default SDMX-component
in PX-Web dotnet
23-04-21 7
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
The links with work packages defined in ESSnet-1
• The Phase II is based on the work in Phase I in order to use the Reference Infrastructure in an NSI in the PC-Axis community
• WP 3 Data Express
23-04-21 8
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
Re-usability and links to SDMX
• Other NSI in the PC-Axis community can use the results• Nordic data model, SQL
• MS-SQL• Oracle• Sybase • MySQL• ODBC
• PC-Axis files
• For international reporting focusing CensusHub• Links to visualisation tools based on SDMX
23-04-21 9
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
The re-usability within ESS• The different activities within the WP 2 will be
documented according to the Eurostat templates and made available within the ESS.
• The deliveries will be available according to the present time plans that is stated for the activities respectively.
• Software within the PC-Axis family will be available on the PC-Axis portal: http://www.scb.se/Pages/StandardNoLeftMeny____314045.aspx
• Software that can be used outside the PC-Axis family will be put on the Eurostat Portal for software downloads together with installation instructions
23-04-21 10
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
Actions in the developing work grouped in areas
23-04-21 11
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
5. Eurostat SDMX infrastructure – test of routines and further development of
SDMX-RI
1. Process descriptions inclusive user manuals at Statistics Sweden
2. Transfer of processes
4. Confidential materials
3. The Data Model and applications
6. Extension of material
Explanation of signs:Activity restricted to cover the CensusHub related processActivity deleted
1. Process descriptions inclusive user manuals at Statistics Sweden• a) Design proposal on the work flow for different
central organizational units and subject matter divisions for a first delivery and work flow for continuous deliveries using Eurostat-RI.
• b) Proposal for the distribution of responsibility between different Objects of Maintenance when it comes to applications in the Infrastructure at Statistics Sweden and the Eurostat-RI. It includes an up date of the Maintenance Specifications.
23-04-21 12
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
• c) User Manual for statistics producers and others. • d) Process Support System• e) Adjustments in the User Manuals and the Process
Support System from Feed Back from the users that has entered new materials etc.
23-04-21 13
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
1. Cont. Process descriptions inclusive user manuals at Statistics Sweden
2. Transfer of Processes
Internally at Statistics Sweden • a) Preparations of tests of the new way of working
for selected test products.
• b) Seminars, information, time table and test together with the producers.
23-04-21 14
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
2. cont. Transfer of Processes To other countries• c) Preparation of User Manuals for the PC-Axis
family of users. As in 1 c but adjusted for the PC-Axis environment and translated into English.
• d) Presentation and information for other countries and Swedish authorities that will use the process developed in Phase IBefore this activity more tests has to be done. The workshop will be carried out in cooperation with ISTAT.
23-04-21 15
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
2. Cont. Transfer of Processes
For other organizations in Sweden responsible for official Statistics – low priority• e) Study of how other governmental bodies in
Sweden will do the reporting to Eurostat in the SDMX era and information concerning the changesStatistics Sweden is local coordinator for Sweden, not only for Statistics Sweden but other organizations.
23-04-21 16
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
3. The Data model and applications
a) Test of PX-Edit concerning Attributes on Cell level and capacity test of PX-Edit, PC-Axis and PX-Web using large census files PX-Edit must be tested at Statistics Sweden when Finland has finalized the development and test. Also including assessment of the User Manual and give feed back to Statistics Finland.
23-04-21 17
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
3. Cont. The Data model and applicationsb) Implementing of attributes on cell level in PX-Web
(dot net version)• Do a study on the amount of work to Develop
Presentation of attributes on cell level in tables and files. The study is based on the Data model and the file format for attributes on cell level. It is desired that the result can be used for the database system on the Statistics Sweden website.
23-04-21 18
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
3. Cont. The Data model and applicationsc) Test and adjustment of the Generic Default
SDMX-format from PC-Axis and check that the same SDMX format is used by other agencies.
• Test of the result from the conversion to Generic Default SDMX format from PC-Axis technical platform. The produced files will be tested in applications that is said to be compatible with the generic format. For instance the Diagram package from Phase I Portuguese and also the eXplorer.
23-04-21 19
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
3. Contin. The Data model and applicationsd) Implementation of the Generic Default SDMX-
component in PX-Web dot netWill be coordinated with the development of the Webb GUI for the database system at Statistics Sweden Website. The system documentation will be updated. Is in the Beta version of PX-Web released in May 2012.
e) Documentation of data model
23-04-21 20
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
3. Cont. The Data model and applications
f) Adjustments for the census Adjustment of publishing software for the CensusHub data. NEW
g) Access to Census conflicts Access to Census server. Can Website access and Web service access the same server without conflicts affecting performance? NEW and deleted since the CensusHub data will reside in a separate server for the Eurostat access.
23-04-21 21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
4. Confidential Statistics• a) Permission issues
Concerns permission inside Statistics Sweden on servers and/or applications. Only those persons working with a certain statistics shall have the permission to watch the statistics. One of the suggestions from Statistics Sweden for the update of the Eurostat RI.
23-04-21 22
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
4. Cont. Confidential Statisticsb) Security issues for confidential statistics• Investigate what recommendation from the
Security organization at Statistics Sweden that is valid for international reporting over a Web Server and Web services.
• Server and Catalogue structure proposals• Encryption issues for Eurostat• Await exercises from the Swedish Defence tests of
Statistics Sweden Website in a related project• Relation to the project for “study of Generated files
and simple web services”.
23-04-21 23
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
4. Cont. Confidential Statisticsc) Further deliveries to other International
Organisations and Confidential Statistics• How is the statistics handled by Eurostat that is
made available for other international organizations? Some data can be confidential.
• Those using encryption today and also those that are sending confidential data today without encryptions? Who are doing what? Check with Eurostat what rules that are valid for third party for encrypted data and Confidential data respectively.
23-04-21 24
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
5. Eurostat Reference Infrastructure – test of routines and further development of the Eurostat RI
a) Complete tests of (the Standard routine and) the Census routine for both the SQL and PC-Axis files in collaboration with Eurostat to get the whole chain showed working. Up date of DSD:s in the Registry are required.
b) Installation and test of new versions of the SDMX-RI; feed-back to Eurostat
c) Meeting in Luxembourg with the developers of the Eurostat RI New
23-04-21 25
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
6. More statistics – low priority
a) Test of different types of statistics for delivery to Eurostat. Will also be included in the maintenance plan
b) b) Possible further development caused by results of test for different statistics
23-04-21 26
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
6. Cont. More statistics – low priority
c) More use cases shall be added to the User’s Manual for the Process- up date, Swedish users Manual and the Process Support System The Unit for National Accounts are interested to get information. They have complex statistics. It is needed that the relevant DSD are made available from Eurostat to make it possible to do the Mapping work. Hitherto very few DSD has been made available. A more Generic User Manual in English is needed for the PC-Axis Family of users.
23-04-21 27
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
The project plan after Kick off meeting 6 September 2011
23-04-21 28
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
ESSnet for SDMX Phase 2, WP 2 PC-Axis SDMX IntegrationProject/activity 2011 2012
Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep1 a Tomas, Ulf, Bodil 3*20 hours1 b Henrik, Ulf, Bodil 3*10 hours1 c Tomas, Ulf, Bodil 35, 35, 10 hours1 d Tomas, Bodil, (Doris) 20, 10 hours1 e Tomas, Ulf, Bodil, (Doris) 10 ,10, 10 hours
2 a Henrik,Ulf, Bodil 3, 3, 3 hours2 b Henrik, Ulf, Bodil 10, 25, 10 hours2 c Ulf, Lars, Thomas 30, 30, 30 hours2 d Lars + ? 56 + 56 hours2 e Lars, Ulf, Bodil, Henrik 20, 20, 10, 10 hours
3 a Lars, Ulf 40, 40 hours3 b Mikael, Petros, Lars, ?, (Doris) 3, 3, 3, 3+7, 7 hours 200 hours???3 c Lars, Petros 10, 10 hours3 d Petros, Mikael 10, 10 hours3 e Petros, Doris, Else-Marie 10, 10, 10 hours
4 a Ulf, Mikael, Petros,(IT4), Lars 10, 10, 10, 10+20, 20 hours4 b Ulf, Mikael, Petros,(IT4), Lars 15, 15, 15, 15 hours4 c Ulf, Bodil 40, 10 hours
5 a Ulf, Lars, Mikael, Henrik 85, 15, 10, 10 hours5 b Ulf, Lars, Henrik 70, 20, 10 hours
6 a Ulf 50 hours6 b Mikael, Petros, Doris, Else-Marie 20, 20, 20, 20 hours6 c Ulf, Lars, Thomas M, (Doris) 25, 25, 25 hours
The project plan April 2012
23-04-21 29
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31
Test of the Eurostat-RI in ESSnet for SDMX Phase 2, WP 2 PC-Axis SDMX Integration, revised April 2012Project/activity 2011 2012
2012-04-26 Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep1 Process descriptions inclusive user manuals at Statistics Sweden1 a Design workflow Ulf and Lars 2x251 b Responsibillity planHenrik, Ulf, Bodil 3*10 hours2 Transfer of 2 c English manuals Ulf, Lars 20, 20 hourspart of 2 d Workshop in Rome 25-27 June2 d Pres for other NSI:sLars + Ulf? 2x10 56 + 56 + 20 hours ?>3 The Data model and applications3 a PX-Edit Attr. Lars, Ulf 40, 40 hours3 b Attr. In PX-Web Mikael, Petros, Lars, ?, (Doris) 3, 3, 3, 3+7, 7 hours 160 hours3 c Test. Generic defaultLars, Petros 10, 10 hours3 d Gen. Def. In PX-WebPetros, Mikael 10, 10 hours3 e Doc. Data models Petros, Doris, Else-Marie 10, 10, 10 hours3 f Adj. For the Census Else-Marie/konsult 50 hours New3 g Access to cens. ConflictsHakim, Ulf, Lars 10+3+3 hours New4 Confidential Statistics4 a Authority issues Ulf, Mikael, Petros,(IT4), Lars 10, 10, 10, 10+20, 20 hours5 Eurostat Reference Infrastructure - test of routines and further developoment of the Eurostat RI5 a Test with Eurostat Ulf, Lars, Mikael, Henrik 50, 15, 10, 10 hours ?>Census fictive dataCensus real data5 b Install. New vers. Ulf, Lars, Henrik 70, 20, 10 hours5 c Meeting RI LuxembourgUlf, Lars 20+2 hours New5 d Test of ISTAT Excel RI webserviceLars 10 hours NewActivities that has been dropped after meeting with Eurostat 26 of January 2012
1 c User manuals Thomas, Ulf, Bodil 35, 35, 10 hours1 d Process supp.syst. Thomas, Bodil, (Doris) 20, 10 hours1 e Adj. Man. Pr.supp. Thomas, Ulf, Bodil, (Doris) 10 ,10, 10 hours2 a Tests products Henrik,Ulf, Bodil 3, 3, 3 hours2 b Intern. Seminars Henrik, Ulf, Bodil 10, 30, 10 hours2 e Study other Sw. Org.Lars, Ulf, Bodil, Henrik 20, 20, 10, 10 hours4 b Security confid. Ulf, Mikael, Petros,(IT4), Lars 15, 15, 15, 15 hours4 c Other int. Org. Ulf, Bodil 40, 10 hours
Process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 30
Process for CensusHub reporting from PC-Axis files
2012-05-26
PC-Axis Directory
4. Mapping Assistant
1. Create file for
loading of Dissemination
data base
Producer
SDMX-ML
Statistical product
production database
Loading
OtherInternational Organisations
NSI
2.PX-Edit
Mapping StoreData Structure
Definitions, codes
mappings etc
External PC-Axis Directory
Fil
Mapping of PX-table to DSD
Listing of Mapping
information for check
6. Publishing to external PC-Axis directory
Copying
Sending querry
3.PC-Axis Main
Module or PX- Web
Check of entered data
5.SDMX
converter/Test Client
SDMX-ML
For Check
2. Loading data and entering metadata
Publishingspecialist
Firewall Firewall
Web hotel
Eurostat91.194.86.67
7a. Pull Requestor
7b.Data Retriever
m.fl.
http://censusweb.scb.se/NSIservice.asmx
Mapping tables in PC-Axis directory With DSD
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 31
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 32
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 33
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 34
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 35
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 36
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 37
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 38
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 39
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 40
Snap shots from software in the process based on PC-Axis files
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 41
Snap shots Test Client
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 42
Snap shots Web Client, main window
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 43
Snap shots Web Client - Selections
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 44
Snap shots Web Client – View Result
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 45
Snap shots Web Client – Show result as a table
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 46
Snap shots Web Client – View Result as Download as a HTML page
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 47
23-04-21 48
Training sesion for PC-Axis SDMX Integration 2011/2012
Process in the CensusHub at Statistics Sweden SQL database
23-04-21
Results from Essnet for SDMX II WP2 PC-Axis SDMX Integration 2012-05-31 49
Process for reporting SQL from SCB the CensusHub without RSS
2012-05-24
Census Statistical SQL
database
4. Mapping Assistant
1. Creating file to load into SuperCross
Producer
SDMX-ML
Census production-
database
Otherinternational organisations
SCB
2.. Load SuperCross for checking and
internal outputs
7b.Data Retriever
etc.
Mapping StoreData Structure
Definitions, codes
mappings etc
CensusStatistical SQL
database
File
Mapping Tables with DSD
Mapping list for check
6. Table
transportCopy
Sending query
5.Test Client/Web Client
SDMX-ML
To check
Eurostat91.194.86.67
Firewall
Q01\A
http://censusweb.scb.se/NSIservice.asmx
Web hotelWith web services
Firewall
7a.Pull Requestor
CensusSuperCross
Loading data
3a. Load SQL
Database
File
3b..Setting attributes
Mapping StoreData Structure
Definitions, codes
mappings etc
Possible internal
Mapping Store??
Mapping Tables with DSD
Copy
Recommended