CZECH STATISTICAL OFFICE | Na padesatem 81, 100 82 Prague 10 | www.czso.cz

Jitka Prokop, Czech Statistical Office

SMS-QUALITY The project and application focused on metadata on quality

European conference on Quality in Official statistics3-5 June 2014, Vienna

2

■ Introduction - aims, features, coverage, current state

■ Architecture - Q-attributes, hierarchical structure, design, preparation, data retrievals and inputs, functionality, stability of values and updates

■ Benchmarking

■ Challenges

Topics

3

Starting points

■ Horizontal way of management

■ Demands for quality reporting & relevant metadata ‚standardisation‘

■ Standardisation of quality reports & adjustments for domain statistics

Tool for managers

■ Semi-interactive cross-cutting overviews about quality of a survey (incl. assessments…)

■ Quality reporting

■ Application using web-browser environment

Reasons and Aims

4

■ Q reporting focused on a survey (of any kind) and groups of surveys

■ Cross-cutting info on quality of statistical process, output data and products

■ Data preferably retrieved from other source databases

■ Data monitoring, comparisons, aggregations, assessment, benchmarking

■ Flexibility of metadata content and possibility of survey’s adjustments

■ Help to improve quality reporting and statistical quality itself

■ Encourage self-assessment, support auditing

Aims and main features

5

■ The application is integrated within the internal SIS and SMS systems

■ Meta-data values retrieved from databases or manually inputted

■ Design & preparation of various quality reports

■ Hierarchical structure (refers to ESQR, GSBPM, DESAP)

■ Public vs. non-public – individual items or complete reports

■ Bilingual (multi-lingual) solution

■ Usual output formats PDF, HTML, XLS, DBF, DOC, not SDMX

■ User roles: admin, owner, editors, viewers (public vs. internal)

Features

6

Other SMS subsystems can provide certain knowledge on quality criteria e.g. accuracy, relevance, accessibility, clarity, timeliness, punctuality.

■ SMS SURVEYS: statistical processes (particular surveys)

■ SMS REQUIREMENTS: management of main user requirements

■ SMS DISSEMINATION, CATALOGUE OF PUBLICATIONS:

dissemination, product quality, info service

in some cases in relation to concrete surveys

Interlinks with other SMS-applications

7

Any statistical process processed or at least with its data stored in the central DWH.

■ Business statistics

■ Social and demography statistics

■ National accounts

■ Price statistics

■ Administrative data statistics.

Coverage

8

Type of info on quality - quantitative and qualitative:

■ Reference metadata

■ Info about process and its phases

■ Schedules

■ Quality performance indicators

■ Calculations

■ Benchmark results

■ Evaluation, (self-)assessments, commentaries

■ Textual, Numerical, Date

Q-attribute (item, meta-information, indicator)

9

■ Basic information (about a survey)

■ User requirements agenda

■ Methodology info

■ Time schedules; Timeliness; Punctuality

■ Statistical process phases

■ Data confidentiality and protection

■ Data sources; Frame; Sample

■ Outputs and dissemination

■ Individual quality criteria (i.e. quality dimensions)

■ Quality performance indicators

Categories of Q-attributes (info on…)

10

Relates to functionality (stability of values)

■ General

■ Statistical survey (key users, methodology, key statistical variables…)

■ Reference year

■ Processing (all reference periods processed or revised at one time)

■ Reference periods

Levels of Q-attributes

11

■ To provide relevant & up-to-date information

■ Validity for certain years, batches (i.e. processings), ref.periods

■ When generating data for new reference periods...

■ Metadata updates on each level

■ How to update the derived Q-Maps

■ Managers informed and decide via the application

■ Keeping history and updates

Updates of metadata structures and values

13

■ Structure (hierarchy): Sections, Sub-sections, Q-attributes

■ Q-Maps: monitoring, benchmarking

General - > Specific -> Survey Q-Maps design, specificationsValue Q-Maps output report

■ Q-Forms: also comparisons and aggregations…

General QM for Q-Forms -> Q-Form -> Value Q-Form

Q-Forms use (not only) Value Q-Maps as the source of data

Q-Maps & Q-Forms

14

Comparisons, aggregations over

■ Statistical variables

■ Reference periods

■ Surveys

■ Years…

Which data

■ Values

■ Benchmark results

Q-Forms - comparisons, aggregations

15

Design of a report

■ General Q-Map -> A type of report. General design, pre-setting of parameters.

■ Specific Q-Map -> A group of surveys.Selection of Q-attributes, way of benchmarking.

■ Survey Q-Map -> A survey Statistical variables, benchmark scales, links to data.

Output report

■ Value Q-Map -> One reference period.

Retrieval, editing, approval of values. Benchmarking.

Levels of Q-Maps - Hierarchy

16

■ Primarily for internal management purposes

■ Benchmarked values: numerical or textual

■ Adjustments of scales (boundaries) for particular surveys

■ Parameters

■ To benchmark or not to benchmark?

■ Manually (each value individually) or Automatically (pre-definitions)

■ Categories’ definition – number of categories, and either definition of boundaries or assignment of values from a nomenclature

■ Categories’ labelling – from a special nomenclature or directly in the app

Benchmarking

17

■ Deeper relations between subsystems

■ Revisions of quality attributes

■ Involvement of domain statisticians

■ Full implementation

■ ESS standard quality reports in SDMX

Challenges

CZECH STATISTICAL OFFICE | Na padesatem 81, 100 82 Prague 10 | www.czso.cz

Thank you for your attention.

Any questions?

jitka.prokop@czso.cz