51
BIASES IN QUESTIONNAIRES Introduction to Medicine II School Year 2010/2011 Class 8, First Year Regent Teacher: Altamiro Costa Pereira Supervisor Teachers: Cristina da Costa Santos and Luís Pinto ICTION OR REALITY

Biases in Questionnaires

  • Upload
    frye

  • View
    64

  • Download
    0

Embed Size (px)

DESCRIPTION

Introduction to Medicine II School Year 2010/2011 Class 8 , First Year. Fiction or Reality ?. Biases in Questionnaires. Regent Teacher: Altamiro Costa Pereira Supervisor Teachers: Cristina da Costa Santos and Luís Pinto. Summary. Introduction Importance of questionnaires Biases - PowerPoint PPT Presentation

Citation preview

Page 1: Biases in Questionnaires

BIASES IN QUESTIONNAIRES

Introduction to Medicine IISchool Year 2010/2011Class 8, First Year

Regent Teacher: Altamiro Costa PereiraSupervisor Teachers: Cristina da Costa Santos and

Luís Pinto

FICTION OR REALITY?

Page 2: Biases in Questionnaires

Summary Introduction

Importance of questionnaires Biases Types of biases Importance of our work

Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

Page 3: Biases in Questionnaires

The importance of questionnaires

People’s Opinion Politics Religion Sports ScienceDemography Medicine …

Introduction

Collected through Have applications

in…

Questionnaires

(1) Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281-91

Page 4: Biases in Questionnaires

Biases

Introduction

Sistematic mistakes or Biases:

The way each question of a questionnaire is written may or may not influence the answer of the inquired person

Errors related to not measuring the situation that is intended to be

measured.

Biased Questionnaire

(2) Choi BC, Noseworthy AL. Classification, direction, and prevention of bias in epidemiologic research. J Occup Med 1992;34:265-71.

Page 5: Biases in Questionnaires

(5)

Question Design

Questionnaire Design

Administration of

Questionnaire

• Insensitive measure

• Forced choice

• Sensitive question

• Vague word

• Overlapping interval

• Ambiguous question

• Juxtaposed scale• Horizontal response format

• End aversion

• Primacy and recency

Types of Biases

(3) Rodríguez MD, Llorca J. Bias. J Epidemiol Community Health. 2004. 58(8): 635-641.(4) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.

Page 6: Biases in Questionnaires

(5) Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Biases

Introduction

“A catalog of biases in

questionnaires”

Page 7: Biases in Questionnaires

… it wasn’t found any article which proves the influence of those biases in questionnaires.

There are several articles identifying possible biases.

BUT…

(6) Sudman S, Bradburn NM. Asking questions: a practical guide to questionnaire design. San Francisco (CA): Jossey-Bass Pulishers; 1982.(7) Foddy W. Constructing questions for interviews and questionnaires: theory and practive in social research. Cambridge (UK): Cambridge University Press; 1993.(8) Choi BC, Pak AW. Bias, overview. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. Vol 1. Hoboken (NJ): John Wiley&Sons, Inc; 1998. p. 331-8. (9) Sackett DL. Bias in analytic research. J Chron Dis. 1979;32:51–63.

Importance of the Work

Page 8: Biases in Questionnaires

Summary Introduction

Importance of questionnaires Biases Types of biases Importance of our work

Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

Page 9: Biases in Questionnaires

Find out if the types of biases such as:

Question Design; Questionnaire Design; Administration of Questionnaire…

…Influence the answers given in a questionnaire.

Aim

Could the way each question is made affect, in a significant way, the answers given by the

inquired person?

Page 10: Biases in Questionnaires

Summary Introduction

Importance of questionnaires Biases Types of biases Importance of our work

Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

Page 11: Biases in Questionnaires

Participants and Methods

“Utilização dos telemóveis pelos estudantes do primeiro ano do Mestrado Integrado em Medicina (MIM) da Faculdade de Medicina da Universidade do Porto (FMUP)”

All the students of the first year of MIM from FMUP

Question Design Questionnaire Design Administration of Questionnaires Other questions

Define the theme of the questionnaire

Define the target

population

Types of biased

questions

Elaboration of two versions

of a questionnaire

Random distribution of questionnaire

s by per of classes

Collect, organize and

analyse

A1

A2

Page 12: Biases in Questionnaires

Participants and Methods

IBM SPSS STATISTICS 1

Collect, organize and analyse

Page 13: Biases in Questionnaires

Participants and MethodsWebsite

http://www.wix.com/fmupturma08/im11

Page 14: Biases in Questionnaires

Summary Introduction

Importance of questionnaires Biases Types of biases Importance of our work

Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

Page 15: Biases in Questionnaires

Questionnaires (9)

Questionnaire A2

Visual Analogical Scale

Questionnaire A1

Page 16: Biases in Questionnaires

Questionnaire A2

Questionnaire A1

QuestionnairesQuestion design - Forced Choice

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 17: Biases in Questionnaires

Questionnaire A1

Questionnaire A2

QuestionnairesQuestion design - Sensitive question

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 18: Biases in Questionnaires

Questionnaire A2

Questionnaire A1

Questionnaires

(10) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.

Questionnaire design - Juxtaposed scale

Page 19: Biases in Questionnaires

Questionnaire A1

Questionnaire A2

QuestionnairesQuestion Design – Forced Choice

5.

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 20: Biases in Questionnaires

Questionnaire A1

Questionnaire A2

QuestionnairesQuestion Design - Vague Word

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 21: Biases in Questionnaires

Questionnaire A1

Questionnaire A2

QuestionnairesQuestion Design - Overlapping interval

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 22: Biases in Questionnaires

Questionnaire A2

Questionnaire A1

QuestionnairesQuestion Design - Ambiguous question

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 23: Biases in Questionnaires

Questionnaire A2

Questionnaire A1

QuestionnairesQuestionnaire Design - Horizontal response format

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 24: Biases in Questionnaires

Questionnaire A1

Questionnaires

Questionnaire A2

Administration of Questionnaire – Primacy and recency

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 25: Biases in Questionnaires

Summary Introduction

Importance of questionnaires Biases Types of biases Importance of our work

Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

Page 26: Biases in Questionnaires

Expected Results

Answers

Questions of Questionnaires

A1

Questions of Questionnaires

A2≠

Page 27: Biases in Questionnaires

Expected Results

The way each question is made affects, in a significant way, the answers given by the inquired

person.

Page 28: Biases in Questionnaires

Summary Introduction

Importance of questionnaires Biases Types of biases Importance of our work

Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

Page 29: Biases in Questionnaires

Results and DiscussionQuestionnaire A1 Questionnaire A2

Total* of inquired students (N) 107 of 118 98 of 120

Percentage of inquired students 91% 82%

* The total number of students who should answer the questionnaires AI and A2 was 118 and 120, respectively. However, only 107 and 98 answered because the others weren’t present when the questionnaires were delivered.

Page 30: Biases in Questionnaires

Questionnaire A2Questionnaire A1

Questionnaire A1 (N=107)

Questionnaire A2 (N=97) p Value

Quão importante é o telemóvel para ti?Median (min-max) 7,4 (0-10) 8,0 (3-10) < 0,001

Tests: Mann-Whitney

Results and Discussion

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 31: Biases in Questionnaires

Questionnaire A2Questionnaire A1

RESPOSTAS VÁLIDAS:Questionnair

e A1(N=85)

QuestionnaireA2

(N=98)p Value

Conseguia fazer a vida normal sem telemóvel. *

N (%)

29 (34%) 32 (33%) 0,834* Neutral questions (“Não sei”) weren’t considered.

Tests: Chi-square

Results and Discussion

21% “não sei”CI95%=[13%,29%]

0%Mmissing answers

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 32: Biases in Questionnaires

Questionnaire A1 Questionnaire A2

Questionnaire A1 (N=107)

QuestionnaireA2

(N=98)p Value

Considera-se viciado no uso do telemóvel

N (%)

21 (20%) 19 (19%) 0,966

Tests: Chi-square

Results and Discussion

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 33: Biases in Questionnaires

Questionnaire A2Questionnaire A1

(10) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.

Questionnaire A1 (N=106)

Questionnaire A2 (N=98) p Value

Quão importante/satisfei

to está com as aplicações do

telemóvel.Mediana (1º quartil – 3º

quartil)

Aplicações Importância

Satisfação

Importância

Satisfação

Importância Satisfação

Funcionalidades

3 (2-4) 4 (4-5) 3 (3-5) 4 (4-5) 0,011 0,655

Duração da bateria

4 (4-5) 3 (3-4) 5(4-5) 3 (2-4) 0,006 0,214

Tarifário 5 (4-5) 4 (3-5) 5(4-5) 4 (3-5) 0,177 0,778

Design 3 (2-4) 4 (3-5) 3(2-4) 4 (4-5) 0,104 0,817Tests: Mann-Whitney

Results and Discussion

Page 34: Biases in Questionnaires

Questionnaire A1 Questionnaire A2

“Concordo/Discordo” not considered Questionnaire A1 (N=45)

Questionnaire A2 (N=97) p Value

Concorda com o desbloqueio ilegal dos telemóveis. *

N (%)

Concordo

32 (71%) 66 (68%) 0,713

Tests: Chi-Square

Results and Discussion

58% “Concordo/Discor

do”CI95%=[49%;68

%]

1Missing answer

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 35: Biases in Questionnaires

Questionnaire A1 Questionnaire A2

Questionnaire A1 (N=107)

QuestionnaireA2

(N=98)p Value

Em média, frequência de envio

de mensagensN (%)

Nunca (0 mensagens diárias)

1 (1%) 0 (0%)

Raramente (1-2 mensagens diárias)

5 (5%) 2 (2%)

Ocasionalmente (3-10 mensagens diárias)

27 (25%) 20 (20%)

Frequentemente (11-30 mensagens diárias)

39 (36%) 61 (62%)

Constantemente (mais de 30 mensagens diárias)

35 (33%) 15 (15%)

Results and Discussion

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 36: Biases in Questionnaires

Questionnaire A1 Questionnaire A2

Questionnaire A1 (N=107)

Questionnaire A2 (N=98) p Value

Mensagens enviadas numa aula teórica

Zero 6 (6%) 1 (1%)

Uma a três 24 (23%) 25 (26%)

Quatro a dez 40 (38%) 37 (38%)

Mais de dez 36 (34%) 35 (36%)

Results and Discussion

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 37: Biases in Questionnaires

Questionnaire A2Questionnaire A1

Questionnaire A1 (N=107)

Questionnaire A2 (N=97) p Value

Uso de telemóveis prejudica as relações

sociais dos jovensN (%)

Sim 29 (27%) 21 (22%) 0,366

Tests: Chi-Square

Results and Discussion

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 38: Biases in Questionnaires

Questionnaire A2Questionnaire A1

Questionnaire A1 (N=107)

Questionnaire A2 (N=98) p Value

Quando utilizas o telemóvel, o que fazes mais frequentemente?

N (%)

Música 4 (5%) 6 (7%)Jogos 0 (0%) 1 (1%)

Chamadas 15 (18%) 14 (16%)Internet 2 (2%) 1 (1%)

SMS/MMS 59 (72%) 67 (75%)Fotos 0 (0%) 0 (0%)

Messenger 0 (0%) 0 (0%)Outros 2 (2%) 0 (0%)

Results and Discussion

23% of missing answers (IC95%=

15,0;31,0)

9% of missing answers (IC95%=

3,3;14,7)[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 39: Biases in Questionnaires

Questionnaire A1 Questionnaire A2

Questionnaire A1 (N=107)

Questionnaire A2 (N=98) p Value

Quais são os três tipos de pessoas com quem mais falas ao

telemóvel?N (%)

Pai 36 (34%) 21 (23%) 0,051

Mãe 78 (74%) 59 (64%) 0,054

Amigos 83 (79%) 71 (77%) 0,397

Namorado(a) 43 (41%) 44 (48%) 0,495Professores 1 (1%) 2 (2%) 0,607

Avós 0 (0%) 0 (0%) -Irmão/Irmã 14 (13%) 18 (20%) 0,298

Primos 4 (4%) 2 (2%) 0,685Colegas de Faculdade

56 (53%) 59 (64%) 0,257

Pessoas que não conheces

pessoalmente0 (0%) 0 (0%) -

Tests: Chi-Square

Results and Discussion

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Page 40: Biases in Questionnaires

It’s very hard to write a biased questionnaire with only one bias per question.

Limitations

Results and Discussion

Page 41: Biases in Questionnaires

We want to understand if there are significant differences between the questionnaire considered biased

and the one considered neutral.

The same inquired person should answer both questionnaires

In what way is that a

limitation?

Limitations

Results and Discussion

Page 42: Biases in Questionnaires

Minimize selection bias

The questionnaires were randomly distributed to the inquired people

Limitations

Results and Discussion

Page 43: Biases in Questionnaires

Limitations

Results and Discussion

The homogeneity of the population:

Same age Same course Same college

Page 44: Biases in Questionnaires

Limitations

Results and Discussion

Lack of sample size in some options

Some statistical tests

Page 45: Biases in Questionnaires

Summary Introduction

Importance of questionnaires Biases Types of biases Importance of our work

Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

Page 46: Biases in Questionnaires

Conclusion

Could the way each question is made affect, in a significant way, the answers

given by the inquired person?

Yes

Page 47: Biases in Questionnaires

Questionnaire A1

ConclusionQuestionnaire A1Questionnaire

DesignJuxtaposed Scale

Visual Analogical Scale

Page 48: Biases in Questionnaires

Conclusion

Some recommendations:

Use visual analogical scales;

Presence of neutral option;

Mixing concepts;

Attention to the order of the options.

Page 49: Biases in Questionnaires

Summary Introduction

Importance of questionnaires Biases Types of biases Importance of our work

Aim Participants and Methods Questionnaires Expected Results Results Discussion Conclusion References

Page 50: Biases in Questionnaires

References[1] Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281-91.

[2] Choi BC, Noseworthy AL. Classification, direction, and prevention of bias in epidemiologic research. J Occup Med 1992;34:265-71.

[3] Rodríguez MD, Llorca J. Bias. J Epidemiol Community Health. 2004. 58(8): 635-641.

[4]Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

[6] Sudman S, Bradburn NM. Asking questions: a practical guide to questionnaire design. San Francisco (CA): Jossey-Bass Pulishers; 1982.

[7] Foddy W. Constructing questions for interviews and questionnaires: theory and practice in social research. Cambridge (UK): Cambridge University Press; 1993.

[8] Choi BC, Pak AW. Bias, overview. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. Vol 1. Hoboken (NJ): John Wiley&Sons, Inc; 1998. p. 331-8.

[9] Sackett DL. Bias in analytic research. J Chron Dis. 1979;32:51–63.

[10] Salant P, Dillman DA. How to conduct your own survey. New York: John Wiley & Sons, Inc; 1994.

Page 51: Biases in Questionnaires

Class 8 Ana Raquel Roca Andreia Freitas Bárbara Vaz Bárbara Neto Joana Costa João Alves Jorge Nogueiro Lucas Lopes Margarida Capitão Maria Durães Mariana Silva Raquel Cardoso

Email: [email protected]