04/22/23 1

Survey and Interview Techniques

Information Systems Research Methods

04/22/23 2

Survey Methodologies

Personal Interviews Group Interviews Phone Interviews Mail-out Questionnaires Hand-out Questionnaires Clip-and-Mail Questionnaires Survey of Secondary Data

04/22/23 3

Survey Methodologies ReasoningThe goal of our research is to describe or

relate the behavior, opinions, impressions, knowledge, etc. of people in situations.

If we cannot directly observe them, their behavior, etc., we must rely on their own (“subjective”) observations of themselves.

Individuals and their actions and opinions are unique, however. No single self-observation can be generalised to everyone.

04/22/23 4

Survey Methodologies Reasoning-2In order to describe the general situation, we

aggregate a series of measurements, observations or judgments about individual (“subjective” impressions of) events.

The “objective” description is the average of all these “subjective” descriptions.

We assume all biases “average out” and compensate for individual differences.

04/22/23 5

Survey Methodologies BackgroundThe strength of a survey is its generalizability. Since

it samples stimuli from the REAL world to be applied in the REAL world, it stands a good chance of having REAL world responses. Issues are these:

Validity: Do the stimuli create the desired responses?Reliability: Are the responses generated in a

dependable way?

04/22/23 6

Survey Methodologies General Method

Select RespondentSample

(Sampling #1)

Create StimulusSample

(Sampling #2)

Motivate RespondentParticipation(Sampling #3)

Select ResponseSample

(Sampling #4)

Induce ResponsesFrom Respondent

To Stimulus*

*Uncontrolled sampling

04/22/23 7

Strengths...and…Weaknesses

+ Friendly, familiar, cheap

+ Relatively quick+ Useful if survey

questions are clear+ Ideal for Descriptive

research+ Uses common skills

- Hard to control- Samples are often very

large- Has hidden flaws that

can be easily ignored- Cannot easily be used

to draw causal inferences

- Retrospective

04/22/23 8

A(1). Personal InterviewsA(1). Personal Interviews Researcher or team member schedules a

meeting with a respondent (subject, participant)

The respondent is asked questions The respondent replies with answers

(responses) Responses are recorded, then later coded

and transcribed if necessary

04/22/23 9

What is an Interview?

@*#?

An interview is a structured conversation between an active agent and a respondent; both are trying to find out what the other thinks, feels, knows, suspects, fears, desires, or respects.

Often the result is quite remarkable! !

04/22/23 10

Interviewing: General Interviewers must be trained Watch out for observer effects

Rapport, empathy VERY important Easy to do VERY badly Each hour “costs” about 3 to 4 hours labor Recording is a problem Useful in uncovering in depth

04/22/23 11

Interviewing: The Four Samples Respondents: Whom you approach for interview

access [sampling frame, permissions]Stimuli [“Schedule”]: Questions you intend to ask

(or sample from)Participation: Times and venues at convenience of

all partiesResponses: What you chose to hear and record

04/22/23 12

Interviewing Procedure• Develop questions pertinent to theory• Create Interview “Schedule”• Pretest Questions on Schedule

• Develop a sampling frame• Sample potential interviewees• Schedule interviews (2 - 4 per day)

• Perform interviews• Write up data• Code/Store/Secure data`

04/22/23 13

Interviewing Issues

Structure of Interview (Typology, constr’n)Flow of InterviewChoice of language and level, jargonDegree of intervention with intervieweeQuestions and open vs. closed questionsUsing tape recordersNon-verbal cues, dress, style, accentPower relationships

04/22/23 14

Degree of Structuring

• Structured: Interviewer is a mechanical questioner, available to clear up ambiguity

• Semi-structured: Interviewer has fixed list but can deviate to follow tangents and get clarification, elaboration

• Unstructured: No preset questions, just follows subject’s ideas

04/22/23 15

Interview Flow Structures

FUNNEL

FAN

TIME

Specific

General

FLIP-

FLOP

Content experts Managers Generalists

04/22/23 16

Language, Level, Jargon

• Your language can tell a lot about you, the research, the answers!

• The criterion is understanding• Select appropriate levels of language for

your subjects• Avoid your own jargon unless interviewing

your own kind of person

04/22/23 17

Open vs. Closed QuestionsOpen: Answers aren’t predetermined, researcher

has no control over answer other than by question“What was your first impression of the interpersonal

skills of your new CIO?”

Closed: Only a limited (usually small) number of possible responses“Which of the following 3 statements best describes

your impression of your new CIO’s interpersonal skills?”

04/22/23 18

Questions:Open Closed

Allows subject to answer in his/her own terms

Makes for more “friendly” interview

May be hard to codeSubject might not know

how to answerUseful for probing

Makes sure subject answers in researcher’s terms

Makes for more systematic interview

Easy to record/codeSubject may answer

incorrectlyMight miss important

responses

04/22/23 19

Intervention

• “Intervention” is a general term implying what you will do for (or with or to) the respondent

• In action research, you are going to make a change in the respondent’s environment

• In other kinds of research you may appear as a change agent or blocker.

• Watch out for “halo effects” wrt technology or development outcomes.

04/22/23 20

Role of Non-Verbals

Non-verbals “modify” your speech and tell subject “who” and “what” you are

Includes these characteristics:Dress Posture ProxemicsAccent Paralanguage EncouragersTone of Voice Sexism/racism Time Mgmt.Mannerisms Facial expression GazeBody Odor Gender Body image

04/22/23 21

Tape RecordersPros Cons

Can theoretically record all responses

Frees interviewer to concentrate on managing the process

Provides a record of what was actually said

Can be used to check your impressions

Sometimes fails or is cumbersome to use

Interviewer might forget to probe or pursue thinking “it’s on tape”

May be intrusive and actually shape responses

May inhibit immediate response formation and hence bias the research.

04/22/23 22

Degree of Structuring

• Structured: Interviewer is a mechanical questioner, available to clear up ambiguity

• Semi-structured: Interviewer has fixed list but can deviate to follow tangents and get clarification, elaboration

• Unstructured: No preset questions, just follows subject’s ideas

04/22/23 23

Interviewing Example

175 handicapped people were interviewed concerning their use of the Internet. Their names were obtained from various societies of the disabled. Questions pertained to when, how, why they used the Internet and what they found difficult to use. Some problems occurred in interviewing deaf people and severely disabled persons. Each interview took about 1.5 hours.

04/22/23 24

Interviewing Diagnostic-1

Street traders are to be interviewed about their impressions of the value of computers in their work. A team of students, in business dress, shows up in Detroit at 17h00 and ask complex questions about hardware and software. What problems should have been anticipated?

04/22/23 25

Interviewing Diagnostic-2

A team of 4 interviewers is trained from the OU basketball team to conduct 100 interviews at a local girls’ prep school concerning students’ need for computer training. One week is allowed for data collection. Interviewers show up in uniform and ask 25 open-ended questions about training needs. What problems might occur?

04/22/23 26

Interviewing Diagnostic-3

You are interviewing managing directors on their views about IT and company strategy. For one interview you show up in sandals and shorts, tardy by half an hour, late on a Friday afternoon. Most of your 18 closed questions concern technology. Do you expect any problems with the data collection?

04/22/23 27

A2. Group InterviewsA2. Group Interviews As with personal interviews, but the

interviewer speaks with a group brought together at the same time

Person who runs meeting is called a “facilitator”.

Meetings may run for an hour or for hours There might be others there: observers or

recorders; it might be video or audio taped

04/22/23 28

Group Interviews: General

Useful, lowers cost Hard to get groups together Facilitation skills are not easy to acquire May help simulate real work environment Beware domination, kidnapping, S&M Locale, interruptions List of attendees is critical

04/22/23 29

Group Interview Benefits

Multistreaming, parallelism

Synergy, group interaction

Simulation of an alm

os t rea l or nor m

a l s itua tion

04/22/23 30

Group Interview ExampleFive groups each of managers,

professionals, and secretaries of five to sixteen people were interviewed in a conference room concerning their perceptions of their role and participation in computerization and how this affected their subsequent use. A facilitator, a recorder, and an observer were present. A video tape recorder was used. The meetings lasted from 2 to 3 hours. Some of the groups were lively, others not. Most corroborated one another’s experiences.

04/22/23 31

A3. Phone InterviewsA3. Phone Interviews• Like personal interviews but…• Much shorter (10-30 minutes MAX)• Level of trust is quite low• Nonverbals are minimized, potential

problem with control, empathy• Cuts down on travel time• Questions must be VERY succinct & clear

04/22/23 32

Phone Interview ExampleA team of student interviewers called 88 CIOs

and conducted twenty-minute interviews on their role in corporate strategy. Each had been mailed a copy of the questions in advance. The interviews had been scheduled with their secretaries. Ten CIOs could not keep their appointments and two were interrupted during the interviews. The interviews were supplemented with biographical data obtained from the secretaries.