Moving from Notions to Numbers: Psychological

MeasurementChapter 4

James A. Van Slyke

Two major challenges to converting notions to numbersEnsuring that research participants are thinking

about the same question that the researcher was thinking about. This is the Judgment Phase

Ensuring that participants are able to translate their internal psychological state into some kind of value on a response scale.This is the Response Translation Phase

Constructing a QuestionnaireThe best choice for selecting a questionnaire is

to use one that already has been established as reliable and valid.

If a suitable measure cannot be found, researchers choose to create their own questionnaire.

It may seem easy, but a lot goes into developing a reliable and valid questionnaire.

Constructing a Questionnaire (continued)

Important steps for preparing a questionnaire:1. Decide what information should be sought.2.Decide what type of questionnaire should be

used (choose between self-administration or researcher administrated).

3. Write a first draft of the questionnaire taking the perspective of your participants

Constructing a Questionnaire (continued)

Important steps for preparing a questionnaire:4. Reexamine and revise the questionnaire after it

is reviewed by experts.5.Pilot Testing- Pretest the questionnaire using

a sample of respondents under conditions similar to the planned administration of the survey or utilize a focus group

6.Edit the questionnaire, and specify the procedures for its use.

Next steps include establishing reliability and validity of the questionnaire.

Constructing a Questionnaire (continued)

Choose how participants will respond:free-response: Open-ended questions allow

greater flexibility in responses but are difficult to code.

closed-response: structured self-report questions such as multiple choice, true-false, and likert scales are quicker to respond to and easier to score, but may not accurately describe individuals’ responses.

Tips for Wording Questions1) Keep it simple- (no more than 20 words)

2) Use informal language

3) Avoid negations (e.g. Does ice cream not make you happy?)

4) Avoid double-barreled questions• (e.g., “Do you support capital punishment and abortion?”)

Tips for Wording Questions Avoid forced-choice items-

(e.g. Are you a republican or a democrat?)

Avoid questions that do not yield any variance-

(Do you think it is wrong to murder someone?)

Avoid leading questions-

(e.g., “Most people favor gun control; what do you think?”).

Tips for Wording Questions

Avoid loaded (emotion-laden) questions-

(e.g., “People who discriminate are racist pigs: T or F”).

Make sure your questions are relevant to everyone in your study-

(How did the movie, “The Notebook,” make you feel?)

Tips for Wording Questions Write multiple questions to assess the same

construct- Are you satisfied with your life?, How do you feel about your life?

Mix it up

Establish a judgmental context- (Considering your current living situation…)

Ease into socially sensitive questions

Tips for Wording Questions Ask sensitive questions sensitively

Guarantee participants’ anonymity

Ordering of Questions

For self-administered questionnaires, place the most interesting questions first to capture respondents’ attention.

For personal and telephone interviews, place demographic questions first to establish rapport with the respondent.

Use funnel questions: Start with the most general questions, and move to more specific questions for a given topic.

Use filter questions: These questions direct respondents to the survey questions that apply directly to them.

AnchorsAnchors- adjectives that lend meaning to the

numbers on a scale

Endpoint anchors and middle anchors