Upload
xerxes
View
43
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Data Quality – UK activities. Iain Macleay Head of Energy Balances, Prices and Publications. 27 September 2013. Contents. Aspects of quality Standard errors Revisions Risk based quality reviews Quality training. Aspects of quality. DECC follow UK statistical practice: Relevance - PowerPoint PPT Presentation
Citation preview
Data Quality – UK activities
Iain MacleayHead of Energy Balances, Prices and Publications
27 September 2013
1. Aspects of quality2. Standard errors3. Revisions4. Risk based quality reviews5. Quality training
Contents
DECC follow UK statistical practice:- Relevance- Accuracy- Timeliness & punctuality- Accessibility & clarity- Comparability- Coherence
Aspects of quality
DECC release data to pre-announced, year in advance, timetable – all releases at 9:30am. Energy Trends – Thursday 26 September;
Thursday 19 December Thursday 27 March Thursday 26 June
Dates set for coming year, by the DECC Chief Statistician – no political interference
If data not released at 9:30 – DECC need to report breech to UK National Statisticians Office
Data released as soon as available
Timeliness & punctuality
Difficult to measure, but …- Sample sizes of surveys published with
information on coverage- Where useful, standard errors published- Weighting to adjust for coverage- Administrative sources used where
appropriate- Check accuracy of recording by comparing
data sources (volume surveys, price surveys, company reports)
Accuracy
Sample sizes and standard errors for Quarterly Fuels Inquiry - published in industrial price methodology note
- As working in policy departments – regular liaison so data meet needs.
- Also try to anticipate their future needs.- Regularly survey of wider user
community to check meeting their needs, every 2 to 3 years – results published on web
- Review content of press notices and channels of communication (tweets etc)
Relevance
- Data presented in consistent format- Helpful commentary drawing users to key
points of interest (even if politically difficult), written independently by the statistics team
- Clear info on contact details of DECC statistical teams
- All info available for free on web- Metadata published – detailed method notes
on web- Some info on revisions published
Accessibility & clarity
Revisions – final consumption annual growth after one quarter
Number of observations (n) 32 First order of autocorrelation (a) of revisions -0.0654Mean of the revisions (m) -0.1074 Adjusted variance of the revisions 0.3531Variance of the revisions (s2) 0.4026 Number of independent observations (n*) 32t-statistics -0.9571 t-adjusted -1.0219t-critical(±) 2.0395 t-adjusted critical(±) 2.0369Test significant at 5% significance level? No Test significant at 5% significance level? No
Mean Revisions = 0.1074- Test used Standard Absolute mean revisions = 0.4842 Test significant? No Is test significant? No
Test for significance of mean revisions
Standard t-test for the revisions Adjusted t-stat for the revisions
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Q1-2005 Q1-2006 Q1-2007 Q1-2008 Q1-2009 Q1-2010 Q1-2011 Q1-2012 Q1-2013
Perc
enta
ge p
oint
s
Revisions in the year-on-year percentage growth rates estimates in final consumption, after 1 quarter
- Monthly data consistent with quarterly, and annual data – revised in line with better more complete information
- Standard geographies used where possible
- Energy balance format used so supply and demand consistent
Coherence & comparability
- Data collections and publications should be reviewed on a regular basis
- Tricky in practice – time consuming activity
- Risk based approach being trialled- Methodically go through checklist- Most activities fairly low risk
Quality reviews
Risk based review template
Sources Methods Systems Processes Quality Users & reputation
People
Census Data acquisition/questionnaire design
System a Data collection & preparation process
Relevance User feedback
People
Admin Coverage of data
System b Results & analysis processes
Accuracy Future user needs
Survey Processing, edit & imputation
Timeliness & punctuality
Reputation
Analysis Accessibility & clarity
Disclosure Comparability
Coherence
Domestic fuel prices inquiry
Sources Methods Systems Processes Quality Users & reputation
People
Survey – 98% sample coverage
Complex detailed survey, many issues including change of tariff structure etc.
System redesigned in 2012
Data validation & editing
Produces bills based on standard consumption rather than actuals
Good feedback received
New person each year as data processed by sandwich student
Good geographical coverage
Spread sheet back-up available
Main system newly developed, but back-up used as double check
Release 12 weeks after end of quarter
Key policy area – so new data needs emerging
Large survey – company 100 tariffs in 14 regions - so much scope for problems.
Analysis
Information published is disclosive, but pre-agreed with former monopoly suppliers
Actions1. Meet companies to improve form filling2. Engage pro-actively with policy to find future
needs
3. Ensure good documentation4. Have sufficient staff trained to use system5. Check data with that from similar surveys6. Check data against firms published annual reports
- How do we ensure good quality statistics
- Well trained staff- Training sessions held focusing on
quality- All staff to attend – take through stages
of statistical value chain- In DECC two statisticians trained up to
train others
Quality training