Upload
doanminh
View
214
Download
0
Embed Size (px)
Citation preview
European approaches -
- for ensuring high quality of European statistics
EMOS Open Day Luxembourg – 25-26.01.2018
Zsuzsanna Kovács
Eurostat
Eurostat
Overview
What is quality and quality in statistics?
Quality framework (fw) of the ESS, its relation with the legal fw
The ES Code of Practice and Quality Assurance Framework of the ESS – 3 levels of quality assurance
The Quality Declaration of the ESS
The Quality function of the ESS
4th level, centralised quality assurance
Eurostat's Quality Reviews' exercise
Quality reporting in the ESS
Quality-conference
From quality assurance to a holistic view of quality
2 Zsuzsanna Kovács
Eurostat
What is quality?
A relative, multi-dimensional concept
Different definitions and approaches exist
The most general and succinct definition: "fitness for purpose/use"
Definitions of international standards:
• ISO 9000: "Degree to which a set of inherent characteristics fulfils requirements"
• ISO 8402:1986: "Totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs"
3 Zsuzsanna Kovács
Eurostat
What is quality in statistics?
Encompasses all aspects of how well statistical processes and outputs fulfil expectations of all of their users and stakeholders =>
Good quality of statistics is:
• Not just meeting user needs but also
• Addressing respondent concerns regarding reporting burden and confidentiality
• Ensuring institutional environment is independent, impartial, objective, comprising sound methodology and cost-effective procedures
4 Zsuzsanna Kovács
Eurostat
Quality framework of the ESS
The 'common' quality framework of the ESS* is composed of (cf. in the Preamble of the ES Code of Practice)
• The European Statistics Code of Practice
• The Quality Assurance Framework of the ESS, as the 'extension' / complement of the ES Code of Practice
• The general quality management principles – quality for the institutions as a whole, QUAL Action 3
The quality framework of the ESS complements the legal framework and they both constitute the robust framework in which high-quality European statistics are developed, produced & disseminated
5 Zsuzsanna Kovács
Legal and quality fw of the ESS – Relation, inter-action, roles
Legal framework – Quality aspects: 7 Principles anchored in 338/TreatyFEU (Primary law) – Those Principles repeated and defined in amended Regulation 223/2009, together with the introduction of the CoP (Art 11) & definition of the output quality criteria and quality reporting (Art 12) – Quality provisions of the sector-specific legislation (Secondary law)
Common Quality Framework: Complements the legal framework: 'dichotomy' (difference in nature)
Zsuzsanna Kovács 6
Eurostat
Legal and quality fw of the ESS – Relation, inter-action, roles
Their 'meeting points': the 7 Principles & the inclusion of CoP, quality reporting and output quality criteria in the Statistical law
Their differences:
• Coverage – the CoP extends the list of Principles and elevates the status of the output quality 'criteria' to 'Principles'
• Approach (stemming from the nature) – in addition to the law's definition, the CoP provides practical guidance, best practices for the implementation of the Principles
Importance of a '1:1 correspondance / matching'? No – they serve different purposes & have their own rationale
Question of the language, wording used – Clarity, exactitude
Zsuzsanna Kovács 7
Eurostat
Eurostat
ES Code of Practice – aim
Ensures public trust and confidence in the independence, integrity and accountability of the statistical authorities as well as in the credibility and quality of the statistics they develop, produce and disseminate
Sets the standards for developing, producing and publishing European statistics
Promotes the application of best international statistical practices, principles and methods by all producers of European statistics
8 Zsuzsanna Kovács
ES Code of Practice – the ESS concept of quality in statistics
• Institutional environment
• Professional independence • Coordination and cooperation • Mandate for data collection • Adequacy of resources • Commitment to quality • Statistical confidentiality • Impartiality and objectivity
• Statistical processes
• Sound methodology • Appropriate statistical
procedures • Non-excessive burden on
respondents • Cost effectiveness
• Statistical output
• Relevance • Accuracy and reliability • Timeliness and punctuality • Coherence and comparability • Accessibility and clarity 9
Eurostat
Zsuzsanna Kovács
Eurostat
ES Code of Practice – mechanism
Self-regulatory
Applies to Eurostat, National Statistical Institutes of the ESS and other national authorities producing European statistics
For each Principle, there are Indicators showing how compliance can be demonstrated
10 Zsuzsanna Kovács
Eurostat
ES Code of Practice – example
Principle 8
Appropriate statistical procedures, implemented from data collection to data validation, underpin quality statistics
Indicator 8.6
Revisions follow standard, well-established and transparent procedures
11 Zsuzsanna Kovács
Eurostat
ES Code of Practice – monitoring
National and Eurostat self-assessments in 2005 http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/code_of_practice/ess_system_compliance
ESS peer reviews in 2006-2008 (1st round) and in 2013-2015 (2nd round) http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/code_of_practice/peer_reviews http://ec.europa.eu/eurostat/web/quality/peer-reviews
Reports and improvement actions per country and Eurostat; Final report to the Parliament and Council
Annual monitoring and reporting of improvement actions (Eurostat & ESGAB); implementation deadline end-2019
12 Zsuzsanna Kovács
Eurostat
Quality Assurance Framework (QAF) of the ESS – aim
3rd level of quality assurance
Applicable across all statistical domains
But it is not part of the Code
Provides methods and tools at institutional and process level
Contains also links to relevant reference documentation
Can also be considered as guidance to compliance assessors
13 Zsuzsanna Kovács
Eurostat
Quality Assurance Framework of the ESS – example
Principle 8
Appropriate statistical procedures, implemented from data collection to data validation, underpin quality statistics
Indicator 8.6
Revisions follow standard, well-established and transparent procedures
Methods of implementation
Guidelines on revision of published statistics exist, are applied and made known to users
Revisions accompanied by explanations made available to users
Quality indicators on revisions are calculated and published
14 Zsuzsanna Kovács
The Quality Declaration of the ESS
The 'crowning' of many years' work –
To demonstrate the competitive advantage of European statistics
What makes them different from other data
Zsuzsanna Kovács
Eurostat
16
Why in the form of a 'Declaration'
Because
It is a self-commitment, a continuous self-engagement of all members of the European Statistical System (ESS)
It builds on all the efforts that the ESS deployed in order to guarantee the development, production and dissemination of high-quality European statistics and services (amended Regulation 223, second round of peer reviews,…)
Zsuzsanna Kovács
Eurostat
17
Its main message: awareness… … and official guarantee: 'ESS made'
A very high level of quality awareness exists in the ESS
All members of the ESS partnership do the same in terms of quality &
The standards are set very high, as defined in the world-class quality framework: European Statistics Code of Practice =>
Therefore, the product: European Statistics equals high quality that can be trusted, is decided upon in a democratic manner & provides high value for money for the societies
Zsuzsanna Kovács
Eurostat
18
Eurostat
4th level, centralised Q assurance – methods & tools, contin. improvement
Labelling
User
satisfaction
surveys
Production
processes
Statistical
products
User
perception
User
requirementsStandards
1- Documentation and measurement
2- Evaluation
3- Conformity
Process
descriptions
Quality reports
and indicators
Quality
assessments
Quality
improvements
20 Zsuzsanna Kovács
Eurostat's Quality reviews – 4th level quality assurance
2nd round of screening / all statistical production processes
In the light of the standards set by the CQF
Planned roll-out: 4-5 years
Main objective: continuous process improvement, independent and objective re-assurance
Basic tool: shortened and pre-filled questionnaire
4 sub-processes:
1. Risk assessment and planning
2. Quality reviews at process level
3. Annual follow-up of improvement actions and reporting
4. Annual process improvement, presentations, training courses
Zsuzsanna Kovács 21
Eurostat
Quality reporting in the ESS
• Data need metadata
Member States have to provide Eurostat with reports on the quality of the data transmitted &
Eurostat has to assess the quality of the data and has to prepare and publish reports on the quality of European statistics
• Quality reports aim at describing and quantifying the quality of the statistics produced, based on defined quality criteria
• In Nov 2015 the ESSC approved the Single Integrated Metadata Structure (SIMS) v2.0 as the ESS standard for quality reporting:
• Euro SDMX Metadata Structure (ESMS): user-oriented
• ESS Standard for Quality Reports Structure (ESQRS): producer-oriented
23
Eurostat
Zsuzsanna Kovács
Eurostat
ESS Handbook for Quality Reports
Describes the structure and content of a standard, detailed ESS quality report
Contains practical examples as well
Explaines the standard ESS quality indicators and their use
Is now being revised, to include the latest developments
http://ec.europa.eu/eurostat/documents/3859598/6651706/KS-GQ-15-003-EN-N.pdf
24 Zsuzsanna Kovács
Quality conference
'European conference on Quality in official statistics'
Biennial event, organised by Eurostat and a NSI of the ESS
Exchange of methods and good practices between statistical offices, international organisations, researchers and academics
Last conference: www.q2016.es (Madrid) 492 participants from 119 institutions and 60 countries
Next conference: 26-29 June 2018, Krakow (Poland)
26
Eurostat
Zsuzsanna Kovács
Eurostat
Quality for the institutions as a whole – difference between managing and assuring quality…
Quality management is the set of systems and frameworks which are in place within an organisation to manage the quality of products and processes – a broad range of activities
Quality assurance is the guarantee of an organisation that the product or service it offers meets the accepted quality standards; according to the ISO, it is a part of quality management – a more technical part, about the core business
27 Zsuzsanna Kovács
Eurostat
…in their respective frameworks
Quality management is implemented via the Quality Management Systems: a set of interrelated or interacting elements that organisations use to direct and control how quality policies are implemented and quality objectives achieved
Quality assurance is implemented via the Quality Assurance Frameworks (QAF): an umbrella for quality work at a statistical office, a place for recording quality concepts, policies and practices and a basis for creating and promoting quality culture
28 Zsuzsanna Kovács
Conclusions
Quality work set up in a robust and coherent system in Eurostat and the ESS
Importance of continuous development and remaining relevant, up-to-date in a fast evolving context
Importance of managing quality within the institution's overall management framework – with a clear view and understanding of the links and inter-relations, scope and nature of the various elements
Zsuzsanna Kovács 29
Eurostat
Thank you for your attention!
Do you have questions or comments?
Visit us:
http://ec.europa.eu/eurostat/web/quality/overview
Eurostat