Embed Size (px)
Citation preview
2013-09-05 SDMX Technical Working Group 1
Introduction to the
SDMX-JSON Draft Format
Sami Airo & Olav ten Bosch
Capacity Building Session – Technical Stream
SDMX Global Conference 2013
OECD
2
Background: JSON
“JSON (JavaScript Object Notation) is a lightweight data-interchange format.”
✓ Text based
✓ Supports Unicode
✓ Documented (www.json.org)
✓ Standardized (RFC 4627)
✓ Language independent (over 30 from ASP to VB)
✓ Supported by tools (all browsers have it built-in)
✓ Used in production (Google, Facebook, Twitter etc.)
3
Background: JSON and SDMX
4
Requirements: Conceptual Design
Developers will be able to write software that requests responses in JSON from SDMX RESTful API and
processes the data and metadata in the JSON response in order to produce useful data visualizations on the
Web.
Database Web
Service
Web
browser
GET request
Response in JSON
Java
Script
5
Requirements: For the Web
For Slow Networks For Mobile Devices
For Diverse Developers For Diverse Users
© S
as
ch
a P
oh
fle
pp
© Z
are
k
6
Requirements: Features
• Must-have
–Conforms to JSON specification RFC 4627.
–Supports the SDMX 2.1 information model.
–Supports the full functionality in the SDMX 2.1 RESTful API for data.
–Supports many different types of visualizations.
–Streamable on the server.
• Linear (the more/less, the better)
–Easy: minimal amount of client code required.
–Fast: few roundtrips, small messages, and fast parsing.
7
Implementation: Basic Message
- Header is for technical information: prepared, sender, ...
- Errors is for error messages.
- Structure contains metadata: dimensions and attributes.
- DataSets contains the actual data.
- Examples: flat grouped
{
“header”: { ... },
“errors”: { ... },
“structure”: { ... },
“dataSets”: [ ... ]
}
8
Implementation: Data + Structure
• Format combines both structure (e.g. dimensions and attributes) and data (observation and attribute values) in the same message.
• Server will join information from different parts of the SDMX IM (e.g. names for codes) and merge it into one message.
• This avoids multiple requests from the client which saves time and simplifies client code.
• Client does not need to process different parts of the SDMX IM and their relationships.
9
Implementation: Metadata Encoding
• Because dimensions and attributes typically take on a limited number of different values we can store them as integers in the dataset and keep the actual values as part of the structure.
• This includes both coded and uncoded attributes.
• The integers map directly to the array indices in the values fields.
• Statistical package R uses a similar method for storing factor (or categorical) variables.
10
Implementation: SDMX Features
• Supports both flat data sets and data sets with series grouping. Also supports optional grouping of common dimension and attribute values.
• Supports multiple data sets with different actions (update, append, delete, informational).
• Supports hyperlinks to other parts of the IM (e.g. data flows).
• Supports optional annotations.
• Includes human readable names and descriptions for data sets, dimension, attributes etc.
• Supports roles for dimensions/attributes (time etc.)
12
Potential Usage Scenarios
13
More Information
• Github Repository
–SDMX-JSON Draft Documentation
–SDMX-JSON Sample Files
–SDMX 2.1 RESTful API tutorial
• Discussion Group
–SDMX-JSON Google Group
