Who responds to website visitor satisfaction surveys?

Ioannis Andreadis, Aristotle University of Thessaloniki

Who responds to website visitor satisfaction surveys?*

Contact: [email protected]

* sponsored by WEBDATANET (COST Action IS1004)

General Online Research ConferenceGOR 13

March 04‐06, 2013, DHBW Mannheim, Germany

Ioannis AndreadisAristotle University of Thessaloniki

Research Questions

1. Is an exit survey sample representative of the population of all website visitors?

– Focus on visitor’s sex, age and education level

2. Can we use website exit surveys to estimate users’ satisfaction level?

– Do we get a better response rate by users who are satisfied and (as a result) do we overestimate the overall satisfaction level?


Voting Advice Applications

• Users read a series of statements on political issues and they express their level of agreement with each of them

• Users can compare their political views with the positions of the political parties


VAA Statement

• Example from Wahl-O-Mat 2009– Railway privatization– German railway

operator Deutsche Bahn should remain entirely owned by the state.

– Answers (Buttons)• Agree, Neutral,

Disagree, Skip


Why a VAA can be used to test the aforementioned hypotheses?

• HelpMeVote.gr is a Greek Voting Advice Application different from Wahl-O-Mat– Before the presentation of the result, users

are asked to fill-in a form with their:– Sex, Age group, Education Level, Vote

Choice, Confidence– Recently added field: Level of Computer and

Internet Experience • Almost all users fill-in this form


Website exit survey

• Questionnaires are attached to outcomes, i.e. each outcome can have only one completed web questionnaire

• Outcomes are session based, i.e. the user needs to close the browser or click a special link to start from the beginning. In any case the user has to answer the whole series of VAA questions before participating to the web survey.


Sample-Population Comparison: Sex

0 20 40 60 80

Female

Male

VAA Population Survey sample


Sample-Population Comparison: Age

0 10 20 30 40

18-24

25-34

35-49

50-64

>64



Sample-Population Comparison: Education

0 10 20 30 40 50 60

Compulsory

Secondary

Tertiary

Postgraduatestudies



Odds ratio (Two logistic regression models)

0,020,02Constant

1,16Confidence by Postgraduate

1,15Confidence by Tertiary

1,11Confidence by Secondary

1,1550-64 by substantive




1,15male by substantive1,361,51Postgraduate (compulsory)

1,181,31Tertiary (compulsory)

1,101,21Secondary (compulsory)

1,371,4950-64 (>64)

1,661,9535-49 (>64)

1,621,8525-34 (>64)

1,441,7118-24 (>64)

1,091,27male


Sex – Vote Choice Interaction on Web Survey Participation

2%

3%

4%

5%

6%

7%

Undecided Substantive Answer

Male Female


Estimated Satisfaction Level (subset of respondents)

• VAAs rank parties according to their proximity/similarity with the user e.g.: Party A 80%, Party B 70%, Party C 30%:

• If the selected party appears first, the user will be very satisfied

• If the selected party appears near the top, the user will be somewhat satisfied

• If the selected party appears near the bottom, the user will be dissatisfied


Sample-Population Comparison: Satisfaction

0 10 20 30 40

Very

Somewhat

Not at all



Conclusions

• Small differences between sample and population regarding Sex, Age and Education – Significant interactions: (Sex and Age)*Vote choice,

Education*Confidence– 6th ISM workshops: Would it be better if I had a

measure of computer literacy? Preliminary results appear to support this hypothesis (but, there are differences between those who start and who finish)

• Satisfaction– Overestimation of satisfied users and underestimation

of unsatisfied users

Thank you!


[email protected]

* sponsored by WEBDATANET (COST Action IS1004)

Technology

Who responds to website visitor satisfaction surveys?