S t a r t Session 4 Methodologies and model questions: households Methodological recommendations for household/individual surveys on ICT usage Christophe

s t a r t

Session 4Session 4

Methodologies and model questions: Methodologies and model questions:

householdshouseholds

Methodological Methodological

recommendations for recommendations for

household/individual household/individual

surveys on ICT usagesurveys on ICT usage

Christophe Demunter

Hartmut Buchow

EUROPEAN COMMISSIONEUROSTATDirectorate D: Single Market, Employment and Social statisticsUnit D-7: Information Society and Services

WSIS Thematic Meeting « Measuring the

Information Society »

7 - 9 February 2005

Geneva

www.europa.eu.int/comm/eurostat/


Information Society Statistics in Internet:EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society

Background of the surveyBackground of the survey

Since 2002, Eurostat coordinates across the EU (and candidate countries) surveys on households’ and individuals’ usage of ICT.

Data collected by national statistical institutes following a Eurostat model questionnaire and methodology

Member States take part on a voluntary basis with financial support from the EU to cover part of the survey costs (2005: 25 European countries)

From 2006 onwards, a European legal act will guarantee full coverage in all EU countries



Contents of the surveyContents of the survey

Model questionnaire (2005) includes questions on: available ICT devices at home; frequency and location of computer and internet use; purpose and nature of internet activities; ICT security; e-skills; barriers to computer and internet access.

Broken down by: age gender employment situation educational level (ISCED) occupation (ISCO) household type urban / rural area, …



Contents of the survey: future stepsContents of the survey: future steps

Move from general questions covering a wide range of topics to more detailed modules on a few selected topics (e.g. for the 2006 survey, a module on e-government may be included)

Lower the frequency of data collection for some questions (e.g. every two years instead of annually)

Remove some questions that have proved to be difficult to collect

Structure of the future surveys (2007 and later) may partly be defined by the new benchmarking indicators for the EU (the list of indicators will be finalised by the European Commission by the end of 2005).



Some general aspectsSome general aspects

Length of the survey: Questionnaire consists of ± 25 questions Survey has ± 125 observation variables Because of routing and filtering, respondents only need to

answer a subset of the questions

Type of questions: Simple: most questions/items only require a “Yes” or “No”

answer

Major strengths: Harmonised data collection across Europe leads to highly

comparable data (also with third countries via OECD) Sample is representative for the whole population (aged 16

to 74)



Scope of the surveyScope of the survey

Questions relating to the available equipment are directed at all households with at least 1 member in the age group 16-74.

All other questions are directed at individuals, where the scope consists of all individuals aged 16 to 74

Lower limit = 16 years Reason: legal objections to surveying children in some countries Disadvantage: -16 is an interesting study group for this domain

Upper limit = 74 years Reason: practical difficulties in collecting data from 74+, sometimes

linked to the survey vehicle Disadvantage: 74+ is an interesting study group for e-Inclusion

?? Future ?? - Expand scope to include more age groups?- Move household questions to individual level?



Data collection method: survey vehicleData collection method: survey vehicle

Stand-alone survey ?or

Questions embedded in an existing survey ?

About half of the countries collect the data in a separate survey

About half of the countries add the ICT usage questions to an existing questionnaire (e.g. Labour Force Survey; Income and Living Conditions Survey; general purpose Omnibus Survey, Micro-Census)

+ ask background variables (age, education, …) only once

+ reduces costs (e.g. travel costs for interviewers)

- very long interviews (high respondent burden, affects quality)



Data collection method: collection modeData collection method: collection mode

Face-to-face interview, telephone interview or postal survey ?

2003 survey Collection mode

Denmark CATI

Germany Postal survey, self administered. Exception Saxony: CATI

Greece CATI

Spain CATI / face to face

Ireland Face-to-face

Italy CATI

Luxembourg Face-to-face

Austria Face-to-face

Portugal Face-to-face

Finland CATI

Sweden CATI

UK Face-to-face

Iceland CATI

Norway CATI



Sample sizesSample sizes

Relatively low sample sizes because of the type of questions (Yes/No answers) and the limited level of detail

Does not always give reliable results for very detailed breakdowns or specific sub-groups (due to low cell frequencies)

Some countries increase sample size for national reasons (e.g. policy need for very detailed regional data)

Net sample sizes for individuals in 2004 range from 1455 (Slovenia) to 41200 (Italy)

Sampling rates for individuals in 2004 range from 0,007% (1 in 14000) in the UK to 0,822% (1 in 120) in Iceland

Average sampling rate for individuals (2004): 0,053% (1 in 1900 persons)



Sample sizes (2004 survey)Sample sizes (2004 survey)

Country Population size (16-74) Sample size Sampling rate

DK (Denmark) 3 933 851 2 898 0.074%

DE (Germany) 61 904 554 8 887 0.014%

EE (Estonia) 1 047 818 1 768 0.169%

EL (Greece) 8 468 905 4 180 0.049%

IT (Italy) 43 917 255 41 210 0.094%

CY (Cyprus) 524 151 3 485 0.665%

LV (Latvia) 1 778 195 6 508 0.366%

LT (Lithuania) 2 575 773 8 312 0.323%

LU (Luxembourg) 333 866 1 536 0.460%

HU (Hungary) 7 750 513 3 538 0.046%

AT (Austria) 6 050 869 6 874 0.114%

PL (Poland) 28 614 320 8 839 0.031%

PT (Portugal) 7 977 239 10 023 0.126%

SI (Slovenia) 1 559 738 1 455 0.093%

FI (Finland) 3 881 449 3 134 0.081%

SE (Sweden) 6 478 282 5 429 0.084%

UK (United Kingdom) 41 559 075 2 973 0.007%

IS (Iceland) 198 730 1 633 0.822%

TR (Turkey) 47 039 185 24 462 0.052%

Total for the above countries 275 593 768 147 144 0.053%



Response and non-responseResponse and non-response

Countries’ experience show that the final response rate is on average around 75%

When aiming at a net sample (= number of elements in the final database) of e.g. 3000 individuals, it may be necessary to select 4000 individuals.

Main reasons for non-response are refusals to participate and non-contacts (respondents cannot be reached)

Risk of biased results: non-users of ICT may be more inclined to refuse participation because they do not feel concerned by the subject of the interview

Importance of imputation (e.g. post-stratification methods to improve the representativeness of the estimates)

Item non-response is low (excepting for a few complex questions)



Response and non-responseResponse and non-response

Overview of response patterns for some participating countries:

Source: 2004 Methodology Reports

(2004 survey) DK LT HU SI FI SE UK IS NO

A. Gross sample size4499 4500 6500 2000 4127 5429 6000 2000 2000

a) Ineligible: out-of-scope n/a 250 - - - - 411 - 109

b) Other ineligible 493 100 143 193 26 9 481 50 21

B. Number of eligible elements = (A – a – b) 4006 4150 6357 1807 4101 5420 5108 1950 1870

c) Non-contact 336 210 1290 142 498 410 629 116 146

d) Refusal 445 223 1034 184 302 589 1427 152 458

e) Inability to respond n/a 2 123 22 51 57 - 20 60

f) Rejected interviews 326 - 50 - 2 - - - -

g) Other non-response n/a 37 4 114 738 79 29 18

C. Net sample size = (B – c – d – e – f – g) 2899 3678 3860 1455 3114 3626 3042 1633 1188

D. Unit response rate = (C / B) 72% 89% 61% 81% 76% 67% 60% 84% 64%



Methodological reportingMethodological reporting

Participating countries are requested to send a methodological report to Eurostat

Since 2004, a template is used which makes it easy to compare the methodologies across countries

Contents: Survey period, reference period, survey vehicle, survey mode, etc. Statistical units used, age groups used, territorial coverage Comparison of national questionnaire to Eurostat model questionnaire Discussion of sampling frame and sampling design Response and non-response (see previous slide) Grossing-up procedures Sampling error: coefficients of variation for selected indicators Annexes: analysis of results, questionnaires, interviewer instructions, …



Methodological ManualMethodological Manual

Eurostat is currently preparing a manual for the NSI’s who already carry out the survey or for NSI’s who wish to collect ICT usage statistics

Improve harmonisation by offering recommended guidelines

First “edition” to be released in Spring 2005 (electronic format)

Contents: Detailed description and explanatory notes for the questions in the

models, definitions used, expected output, etc. General outline of the survey: reference and survey periods used,

scope of the survey (target population), etc. Methodological recommandations in terms of sampling design, data

collection methods, questionnaire structure, data processing, estimation methods, non-response treatment, quality measurement, etc.



ICT usage data for the European UnionICT usage data for the European Union

Since October 2004, the results of the ICT usage surveys are available via the New Cronos database which can be accessed free of charge on Eurostat’s website:

Go to: www.europa.eu.int/comm/eurostat

Then go to « Data » - « Database » - « Themes »

Then select « Information Society Statistics »

under one of the following themes:

Population and social conditions

Industry, trade and services

Science and technology

Documents

S t a r t Session 4 Methodologies and model questions: households Methodological recommendations for household/individual surveys on ICT usage Christophe