SDMX at the IMFProgress Report

Expert Group on Statistical Data and Metadata Exchange (SDMX 2007),

Geneva, May 8-11, 2007Patrick Hinderdael, Economic Systems, TGSCathy Wright, Statistical Information Mgt, STA

Data Dissemination

Joint External Debt Hub (JEDH) Consolidates external debt and foreign assets data from creditor and

debtor sources Jointly with BIS, OECD and IBRD Posted quarterly on site managed by IMF and registered in OECD’s

registry for collection by IBRD and dissemination through www.jedh.org

IFS Exchange Rates Posted monthly on site managed by IMF for collection by OECD and

loading into OECD.Stat Investigating use of OECD’s SDMX web service to disseminate from

IMF.Stat (IMF’s implementation of OECD.Stat data warehouse) BOP Data Structure Definition

Standardizing in consultation with ECB

LEGEND

1. Data consumer requests statistical data through a web page.

2. Web page receives the request, analyzes it and reformats the request in SDMX query language.

3. Web page contacts SDMX web service (local or remote) and submits the query.

4. SDMX web service receives the SDMX query, examines it and validates it against schema definition.

5. SDMX translates the SDMX query into XML query.

6. SDMX web service contacts Dataset Browser (DB) XML web service.

7. DB XML web service receives the request, examines it and validates it against schema definition.

8. DB XML web service transforms the XML request to SQL language.

9. DB XML web service connects to the SQL database and submits the query.

10. DB XML service receives the dataset.

11. DB XML service transform the dataset and packages it in XML language.

12. DB XML service sends the file in SOAP.

13. SDMX service receives XML file.

14. SDMX transforms the XML into SDMX language.

15. SDMX service responds user’s request by giving SDMX message back.

Data Request Fulfillment through SDMX (current state)

Server (outside Firewall) WEBSWN34D DBSWN11D

SDMX web service

(4, 13)

SDMX to XML translator

(5, 14)

Database

Data browser

(7, 8, 11)

9

10

6

12

1

16

2-3

15

Data Receipts

GESMES (SDMX-EDI) Processing Data from BIS, ECB, Eurostat and 4 central banks

ABS NAWWE data Identified non-standard syntax with help from Dimension-EDI Contacted OECD and ABS to clarify position Need “Standards Certification Authority”?

OECD MEI and other OECD data Potential use to replace existing csv file processing.

Possibility to leverage Eurostat’s SODI-related data

LEGEND using ABS NAWWE data sheet as an example: Step 1 converted the supplier’s (ABS) data (NAWWE) into an SDMX-ML file. (This is what the IMF received). Step 2a IMF used a conversion tool created by Dimension-EDI (Chris Nelson and Arofan) to convert it to a CSV file. In fact, Chris helped us in converting the file because the ABS SDMX-ML file had some errors. Chris also provided a list of these errors and suggested fixes.Next Steps Step 3 Use a code list from ABS to map their TS codes, country lists, etc. to our codes and create an EDF-XML file that will be automatically uploaded into EDF in Step 4. Note that the mapping will be done only once. Step 2b A general solution would be to develop a tool (BizTalk has been recommended by our technical staff) that will use the mapping information and convert SDMX-ML directly to EDF-XML

What is needed for steps 3 and 4. We should:

1) ask ABS to correct their SDMX file based on the recommended fixes from Chris Nelson.

2) ask ABS to provide us their code list so that we can map their codes to ours.

Collecting Data Through SDMX-ML Forms(A schematic presentation)

Supplier’s Database

SDMX-ML

2a

2b

CSV

EDF-XMLEDF

431

Content-Oriented Guidelines

The IMF was an important contributor to the March 2006 draft version of the SDMX Content-Oriented Guidelines (COG).

In particular and as a result of its experience working with large amount metadata from countries subscribing to the Special Data Dissemination Standard (SDDS) and participating to the General Data Dissemination System (GDDS), the IMF provided inputs to the development of the high-level concepts that are proposed in the COG for facilitating the exchange of reference metadata under the SDMX.

MetaData

Need Metadata Structure Definitions for SDDS and GDDS metadata

Plans to collect and disseminate GDDS and SDDS metadata in SDMX-ML

Issues and Questions

Investigating the use of long-term strategic tools that would fit with IMF’s Microsoft-based data systems such as BizTalk, Windows Communication Foundation (WCF) and SQL 2005’s System Integration Services (SIS)

SDMX Registry: centralized or decentralized? How will the latter work?

What kind of implementation of SDMX is recommended? push, pull or both?