© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics...

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Business Surveys – Testing Strategies in the FSO (GER) QUEST WORKSHOP 2007, Ottawa 23th-

26th April Karen Blanke

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 2

Survey on Trade Statistics

Sample size: 55.000 enterprises

Data collection mode: paper/pencil or electronical questionnaire, self-administered

Topics: Structure and amount of employees Stock and purchase of goods Investment and subsidies Turnover of the financial year

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 3

Survey Form: Trade Statistics

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 4

Feedback on Trade Statistics:

By the enterprises: Burdensome Useless – does not reflect reality Relevance of data

By the public: Timeliness Inconsistency with other findings

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 5

Basic problems of business surveys:

Availability of data (record formation) Willingness to participate Respondent selection Retrieval and judgement (knowledge of

data sets and access) Reporting – (statistical and accounting

categories) Authority to provide data

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 6

Testing principles:

Aim of testing: No total revision Resources: 2 collaborateurs Focus:

Check different perspectives of users Use what is available Focus on respondents‘ views

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 7

Conceptual frame of testing:

Conceptual testing: Comparison of definitions

(accounting system and official statistics)

Quantitative methods: Evaluation of data sets Main Hotline requests

Qualitative methods Discussions with experts and data

entry staff Company-site-visits

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 8

Preliminary findings (I):

Data sets: comparison raw data vs. final data:

Highest rate of correction does not correspond with problems noticed by respondents

Over and underestimation of corrections

No evidence of the reason of unreliable data

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 9

Preliminary findings (II):

Problems encountered via callbacks and hotline:

Interrelated errors Implausible data Via hotline: case-specific problems,

less general problems

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 10

Preliminary findings (III):

Discussion with experts: Experts are quite convinced of their

concepts They provide feedback on general

concepts, need of specific items Provide helpful knowledge on

organisational issues, layout aspects for data entry

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 11

Preliminary findings (IV):

Discussion with data entry staff: Sensitive on layout issues Order and structure of questionnaire Accepted level of detail by

respondents Check of interrelated errors

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 12

Preliminary findings (V):

Company-site-visits: Companies are difficult to recruit and

less is more Main problem: Companies are

overloaded by surveys Not only companies are different but

also the respondent in the companies

Instructions are rarely in use; respondents prefer their own definitions

However: Instructions and definitions are needed

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Folie 13

Discussion

Relevance of quantitative and qualitative testing in business surveys

What are the most helpful business survey testing methods?

Recruitement of enterprises Laboratory tests in business surveys Aspects of visual design, e.g. 3 columns

design (NL) Treatment of instructions (length,

location)

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Thanks for your attention

Karen.Blanke@destatis.de