© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Business Surveys – Testing Strategies in

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics

Statistisches Bundesamt

Business Surveys – Testing Strategies in the FSO (GER) QUEST WORKSHOP 2007, Ottawa 23th-

26th April Karen Blanke



Folie 2

Survey on Trade Statistics

Sample size: 55.000 enterprises

Data collection mode: paper/pencil or electronical questionnaire, self-administered

Topics: Structure and amount of employees Stock and purchase of goods Investment and subsidies Turnover of the financial year



Folie 3

Survey Form: Trade Statistics



Folie 4

Feedback on Trade Statistics:

By the enterprises: Burdensome Useless – does not reflect reality Relevance of data

By the public: Timeliness Inconsistency with other findings



Folie 5

Basic problems of business surveys:

Availability of data (record formation) Willingness to participate Respondent selection Retrieval and judgement (knowledge of

data sets and access) Reporting – (statistical and accounting

categories) Authority to provide data



Folie 6

Testing principles:

Aim of testing: No total revision Resources: 2 collaborateurs Focus:

Check different perspectives of users Use what is available Focus on respondents‘ views



Folie 7

Conceptual frame of testing:

Conceptual testing: Comparison of definitions

(accounting system and official statistics)

Quantitative methods: Evaluation of data sets Main Hotline requests

Qualitative methods Discussions with experts and data

entry staff Company-site-visits



Folie 8

Preliminary findings (I):

Data sets: comparison raw data vs. final data:

Highest rate of correction does not correspond with problems noticed by respondents

Over and underestimation of corrections

No evidence of the reason of unreliable data



Folie 9

Preliminary findings (II):

Problems encountered via callbacks and hotline:

Interrelated errors Implausible data Via hotline: case-specific problems,

less general problems



Folie 10

Preliminary findings (III):

Discussion with experts: Experts are quite convinced of their

concepts They provide feedback on general

concepts, need of specific items Provide helpful knowledge on

organisational issues, layout aspects for data entry



Folie 11

Preliminary findings (IV):

Discussion with data entry staff: Sensitive on layout issues Order and structure of questionnaire Accepted level of detail by

respondents Check of interrelated errors



Folie 12

Preliminary findings (V):

Company-site-visits: Companies are difficult to recruit and

less is more Main problem: Companies are

overloaded by surveys Not only companies are different but

also the respondent in the companies

Instructions are rarely in use; respondents prefer their own definitions

However: Instructions and definitions are needed



Folie 13

Discussion

Relevance of quantitative and qualitative testing in business surveys

What are the most helpful business survey testing methods?

Recruitement of enterprises Laboratory tests in business surveys Aspects of visual design, e.g. 3 columns

design (NL) Treatment of instructions (length,

location)



Thanks for your attention

[email protected]

Documents

© Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Business Surveys – Testing Strategies in