Activities to reduce response burden:

development of questionnaires and

electronic data submission

environment eSTAT based on

feedback from respondents and data

processing phase

Epp Karus

Statistics Estonia

Head of Department

Data Processing and Registers Department

Organisational structure

The organisational structure of SE is based on the Generic Statistical

Business Process Model (GSBPM) 4.0

Work organisation is fully centralised

In autumn 1993, marketing and dissemination, and in 2004, data

collection were centralised

A major re-engineering of the statistical production process took place

in October 2013, when data processing and data warehouse activities

were centralised

Minor changes were made in March 2015 and in October 2017.

Structure http://www.stat.ee/contacts

Statistics Estonia

Implementation of GSBPM in SE

The main reason for such a centralised work organisation is to

increase resource efficiency

Consolidation of similar functions has contributed to standardisation

Standardised tools are implemented in different parts of the statistical

information system

The use of GSBPM has raised the awareness about statistical

production and processes in SE

GSBPM is used for analysing the efficiency and similarities /

differences between statistical activities and for making other analysis

about the statistical programme

Better horizontal integration across statistical domains has been

achieved

Statistics Estonia

5.2 Classify

& code

5.3 Review,

validate & edit

5.5 Derive new

variables &

statistical units

5.4 Impute

5.6 Calculate

weights

5.7 Calculate

aggregates

5.8 Finalize

data files

5.1 Integrate data

1. Specify

Needs2. Design 3. Build 4. Collact 5. Process 6. Analyse

7. Disse-

minate8. Archive

3.1 Build data

collection

instrument

3.2 Build or

enhance process

components

3.4 Test

production system

3.3 Configure

workflows

3.5 Test statistical

business process

3.6 Finalize

production system

6.1 Prepare draft

outputs

7.1 Update

output systems

6.2 Validate

outputs

7.2 Produce

dissemination

products

6.4 Apply

disclosure control

7.4 Promote

dissemination

products

6.3 Scrutinize

& explain

7.3 Manage release

of dissemination

products

8.1 Define

archive rules

8.2. Manage

archive repository

8.4. Dispose of

data and associated

metadata

8.3. Preserve data

and associated

metadata

9.1. Gather

evaluation inputs

9.2. Conduct

evaluation

9.3. Agree

action plan

6.5 Finalize

outputs

7.5 Manage user

support

1.2 Consult &

confirm needs

1.4 Identify

concepts

1.3 Establish

output objectives

1.5 Check data

availability

2.1 Design outputs

2.2 Design variable

descriptions

2.4 Design frame

& sample

methodology

2.3 Design data

collection

methodology

2.5 Design

statiatical processing

methodology

2.6 Design

production

systems & workflow

4.1 Select sample

4.2 Set up

collection

4.4 Finalize

collection

4.3 Run

collection

1.6 Prepare

business case

9. Evaluate

Quality Management / Metadata Managemen

1.1 Determine

needs for

information

Statistics Estonia

Marketing and Dissemination Department

Statistics Departments

Data Warehouse Department

Data Processing and Registers Department

Data Processing and Registers Department

General Department

Metadata Department

Lean tools and Agile methods in SE

In 2016, SE introduced Lean tools and Agile methods to define

the value stream and make value flow better. Our objectives are:

To achieve better coherence between strategic, tactical

and operational planning

To reduce work-in-progress through a systematically

arranged management system and sharing of

competences

To improve work flow in the value stream and information

flow between employees

Statistics Estonia

Lean internal audit results, May 2017

Statistics Estonia

0

1

2

3

4

5

6

1. Visualize

2. Limit Work-in-Progress

3. Manage Flow 4. Feedback Loops

5. Continuous Improvement

Respondents’ feedback collected with

the Promoter Index (PI) questionnaire

The PI questionnaire is used to get feedback from

respondents (https://en.wikipedia.org/wiki/Net_Promoter)

PI has been measured in SE since 2014, using the web

application recommy.com (https://app.recommy.com)

Collected information is used to: receive input for the strategic goal “Reduce respondents’ perceived

burden”

get feedback from respondents about eSTAT, questionnaires, statistical

work and the reputation of SE

meet the respondent needs and develop eSTAT, questionnaires and

communications Statistics Estonia

Questions of the PI questionnaire

and coding of respondents’ comments Questions

Q1: How likely is it that you would recommend to submit the

questionnaire XXX using the electronic data submission environment

eSTAT to your friend or colleague?

Q3: How easy was the completing of this questionnaire in your opinion?

The respondent has the opportunity to comment on his/her responses

Comments are coded into three groups: eSTAT, questionnaire and

reputation

Each category is divided into several development needs and positive

feedback

Statistics Estonia

Comments to PI questionnaire

by topic, 2016

Statistics Estonia

0

100

200

300

400

500

600

eSTAT - Development need

eSTAT - Satisfaction Questionnaire - Development need

Questionnaire - Satisfaction

Reputation - Development need

Reputation - Satisfaction

Contact Centre of Respondents (CCR)

CCR was created in SE in January 2006

From the CCR, one can quickly obtain information about the

respondents’ problems and react to them

The purpose of the CCR is to assist respondents with submitting data

and to provide solutions to problems encountered by respondents of

both social and economic surveys

CCR provides consultation in Estonian, but also in English and

Russian; consultation is provided by telephone and e-mail

To help the CCR consultants to answer respondents’ questions,

guidelines for every questionnaire of SE and a list of answers to

frequently asked questions are compiled

Statistics Estonia

Topics of incoming calls

In 2017, we started to code the topics of incoming calls to

the CCR

This gives us information about the main problems of

respondents and the number of similar problems

The main problems of respondents are shared with the

survey manager, questionnaire team and main users of

data collection environments

If possible, the respondents' proposals are taken into

account in developing and maintenance of questionnaires

or eSTAT

Statistics Estonia

Topics of incoming calls, May−August 2017

Statistics Estonia

Consultation on eSTAT questionnaires 67%

Consultation on eSTAT usage

22%

Consulting social surveys'

respondents 1%

Consulting on finding information

on the website 10%

Architecture of the information system

Dissemination

Statistical

registers

Metadata

system

Data

collection Processing Statistical

analysis Persons

Administrative

registers

Users

Economic

entities

Data

Warehouse

iMETA

VVIS eGeostat

SRS

VAIS eSTAT PX-Web /.Stat

Census-HUB

KUNDE

Analyse

Statistics Estonia

Data processing system VAIS SE is implementing the template based data processing system

VAIS

VAIS was first introduced in 2012 to process the 2011

Population and Housing Census data

Today, the data of about 3/4 questionnaires are processed in

VAIS

This has created opportunities for standardising the data

processing of statistical activities and calculating standardised

quality indicators in the final microdata database

Quality indicators are calculated automatically for all statistical

activities that are processed in VAIS Statistics Estonia

Analysing of microdata edits

For questionnaire developing, we analyse microdata edits

We focus on the variables with the biggest number of edits

The need for editing indicates that the respondent has had

difficulties with collecting and submitting the data or does not

understand what data is required to be submitted

Problematic questionnaires are evaluated in collaboration between

the survey manager, questionnaire team and dataset team

At the questionnaire evaluation meeting, the participants analyse

how to make questionnaires more understandable to respondents

and more compatible with the accounting data of enterprises

Statistics Estonia

Designing and building questionnaires

Before designing and building questionnaires for the new survey

period, all available feedback information is studied by the

questionnaire team

Available sources are:

Complaints and proposals of respondents from the PI

questionnaire

Complaints and proposals of respondents to the CCR

Feedback from the questionnaire evaluation meeting

Based on respondents’ feedback to the PI questionnaire, 21

statistical questionnaires were developed for 2017

Statistics Estonia

Main developments in questionnaires wording of error messages

variable explanations

added pre-fillings, added auto-sums

decrease of variables

changed table structure

translation into Russian

changed coding list

In 2017, we shall pilot the calculation of feedback information (ratios) into

questionnaires based on the data reported with questionnaires to motivate

enterprises to submit data

In 2018, all questionnaires shall be translated into English to help non-

Estonians and e-residents to report data

Statistics Estonia

eSTAT developments

The main user of eSTAT regularly analyses the respondents’

feedback on eSTAT

Smaller improvements can be implemented quickly. As regards

bigger improvements, we are dependent on the financing of IT

investments

Currently, an eSTAT design project and two eSTAT

developments with financing from Eurostat grants are ongoing

Planned archiving of eSTAT’s older data is planned to be carried

out in October 2017 – this will speed up eSTAT and help

respondents to find the right questionnaire more easily

Statistics Estonia

Regular notifications to respondents

At the end of each year, SE informs enterprises of their obligations in

the forthcoming year

Upon first contact, we describe more precisely the principles of

statistical production and the importance of data provision, and also

provide instructions on creating the main user on eSTAT

During the year, we specify the activities of new enterprises and add

new units to data collection based on sample selection criteria

We inform new enterprises of samples of their data submission

obligation on the 12th day each month

Information on data submission is available at http://www.stat.ee/en

Statistics Estonia

Content of eSTAT notifications

In 2017, the content of the automated notifications was

changed

The most important message is included at the beginning of

the letter; the letter is easy to read and is of moderate length

At first, we are informative and later assertive. The text of the

letter is more strict after the deadline has expired

We only inform respondents about enforcement and penalties

7 days after the expiration of the deadline, as we do not want

to alarm conscientious respondents

Statistics Estonia

Timing of eSTAT automated notifications

In 2017, we changed the number of eSTAT automated

notifications: one notification sent out before the deadline was

cancelled and the last automated letter is sent out earlier than

before

New eSTAT automated notifications are sent out by e-mail

-5, +3, +7, +20 days from the data providing deadline

From October 2017, the eSTAT automated notifications are

not sent out at weekends or public holidays. Enterprises were

very dissatisfied with receiving letters from SE at weekends.

Statistics Estonia

Promoting usage of eSTAT

Although the time spent on data collection and contacting

respondents has decreased year by year, it accounts for a

considerable share of the time spent on statistical activities

In 2017, we shall pilot promoting higher eSTAT usage to

data providers for reporting data

When contacting enterprises, we first recommend the use

of eSTAT; secondly we assist with submitting data via

eSTAT and thirdly, we receive data by phone

Finally, we ask about the main barriers to not using eSTAT

Statistics Estonia

Trainings for respondents

SE organizes regular trainings for respondents

We provide trainings on the eSTAT environment and various

most complicated questionnaires

At the beginning of the year, special training to new data

providers is organised

The main user of eSTAT is responsible for eSTAT topics and

the survey manager for questionnaire topics

We inform potential enterprises interested in training about

future trainings via e-mail. Registration for trainings takes place

on the website of SE

Statistics Estonia

THANK YOU

Statistics Estonia

Statistics Estonia 21.09.2016