Questionnaire Validation

7/30/2019 Questionnaire Validation

1/29

Bony Wiem LestariDepartment of Epidemiology and Biostatistics

2010


2/29

A tool for collecting information to describe,compare, explain, knowledge, attitudes, behaviors,and/or demographic characteristics on a particulartarget group


3/29

Self administered (mailed or personal contact) In person (face-to-face) Telephone interviews


4/29

Interviewer can clarify unclear questions Literacy is not required Interviewer can collect more complex answers and

observations Interviewer can minimize missing and inappropriate

responses Interviewer can prevent respondent

from answering out of sequence


5/29

Much less staff time is required, with no dead timewaiting for potential participants

Less potential for observer bias Anonymity may minimize social desirability bias, and

encourage more honest responses to sensitivequestions

Computer-assisted telephone interviewing (CATI) byrandom-digit dialing (RDD) combines advantages ofboth verbal and written methods.


6/29

1. Pre-coded, single-choice questions2. Open-ended questions3. Presence-absence questions4. Rank-ordering questions5. Likert-type questions6. Index development


7/29

Pre-coded, single-choice questions askrespondents to indicate which one categoryapplieso Answers are pre-coded for easy data entry

Categories should be mutually-exclusiveo i.e., no categories overlap with one another


8/29

4. What year are you in?Freshman 1 Sophomore 2 Junior 3 Senior 4


9/29

Open-ended questions do not have pre-setanswers. Excellent way to explore new areas

Use open-ended questions when:o Too many response categories (year of birth)o You dont wish to impose categories on respondentso Really consulting respondentso Qualitative source of quotationso Determining appropriate categorizationo You want a change in pace for respondents


10/29

Minimize the number of open-ended questionsbecause they:o Are time-consuming to codeo May generate responses that are inconsistento Are more likely to be left blank

May decrease response rateo Takes more time to complete the questionnaireo Use opinion-seeking questions sparingly


11/29

23. What is the one thing you would like to seechanged at the university?___________________________________________

___________________________________________


12/29

Presence-absence questions ask respondents tocheck off which items in a list do or do not apply to

them

Less commonly used than other types of questions


13/29

23. Have you ever had contact with a physicallyhandicapped person in any of these groups? (Circle toindicate yes or no for each group)Yes No

Community ------------- 1 0Family ----------------- 1 0Relatives --------------- 1 0Elementary school ------- 1 0High school ------------ 1 0University class --------- 1 0

As a co-worker --------- 1 0


14/29

Rank-order questions ask respondents to indicatean ordering of response items, usually from mostpreferred to least preferred

Must be done with great careo Ask for only three most important itemso Must make instructions explicit

These types of questions should be avoided orminimized because they take time


15/29

31. Rank-order the three most important characteristics you want inthe job you make your lifes work. (Place a 1 beside the mostimportant one; a 2 beside the second most important one; and a 3beside the next most important one.)

High salary.. _____Security... ____Continued interest.. ____Power.. ____Prestige ____Excitement.. ____


16/29

Likert-type questions ask respondents to indicatehow much they agree or disagree with a statement

Response options originally included: strongly

disagree, disagree, are undecided or neutral,agree, strongly agree


17/29

In the following, circle a number to indicate the extent to which you agreeor disagree with each statement:52. I believe capital punishment represents the most effective deterrent tomurder.Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree53. I believe a murderer can be rehabilitated to become a resonsible, functioningmember of society.Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree


18/29

Attitude:An enduring disposition to consistentlyrespond to various aspect of the world,including persons, events and objectsTypically seen as having three components:

Cognitive Affective

Behavioural


19/29

The term scaling refers to procedures forattempting to determine quantitative measures ofsubjective and sometimes abstract concepts. It is

defined as a procedure for the assignment of

numbers to a property of objects in order to impartsome of the characteristics of numbers to the

properties in question.


20/29

Visual analogue (thermometer) scales Frequency-weighted multi-symptom indices

o I-PSS score Multi-symptom checklists

o Charlson/Elixhauser comorbidity lists Grading or ranking Likert (summative) scales :

Respondents indicate their attitudes by checking how stronglythey agree or disagree with statements


21/29

Adjectival scale (variation of Likert) Semantic differential scale:

An attitude measure consisting of a series of seven-point bipolar rating scales allowing response to a

concept

Guttman (cumulative) scales


22/29

Conceptualization:abstract construct conceptual definition operasionalization - indicatorConstructing

TestingAnalysisRevising


23/29

The quality of a measurement: validity andreliability

Validity:

the extent to which a test measures what it isintended to measure

Free of systematic error Accuracy


24/29

Reliability: The degree to which a variable has nearly the

same value when measured several times Free of random error (chance) Precision Also called reproducibility, consistency Assessing precision (reproducibility of repeated

measurement):o Within between observero Within between instruments


25/29

Reliability coefficient ~ correlation coefficient (01)

A questionnaire is reliable when the minimumreliability coefficient is 0.7


26/29

External consistency: Test-retestAlternate form (paralel form)

Internal consistency:o Split-half (Spearman Brown correction)o Kuder Richardson (KR-20, KR-21)o Guttmano Alpha Cronbach


27/29

Criterion validity (predictive and concurrent)depending on whether the criterion refers to a currentor future assessment Construct validityassembly evidence to support or refute a complex

scientific theory and to show under whatcircumstances it holds true


28/29

Content validityo refers to comprehensivenesso how adequate the sampling of questions reflects the

aims of the index that were specified in the conceptualdefinition of its scope


29/29

prior to using a questionnaire it has to beestablished that the instrument is valid andreproducible in the context (i.e. population, settingand study design) in which it is going to be

employed. conceptually comparable (conceptual equivalence) developing a new questionnaire is very time-

consuming burden on respondents practical aspects of the study

Documents

Questionnaire Validation