ZEKI SIMSEK JOHN F. VEIGA - University of Crete

ORGANIZATIONALRESEARCH METHODSSimsek, Veiga / ELECTRONIC SURVEYTECHNIQUE

ZEKI SIMSEKJOHN F. VEIGAUniversity of Connecticut

Even though e-mail is the most widely used computer-mediated communication

medium, its considerable potential as a survey technique has received little atten-

tion from management scholars. Using a three-dimensional framework focused

on sampling issues, nonsampling errors, and comparative performance, the

authors review and integrate previous research on the electronic survey technique

and provide an assessment of the comparative trade-offs vis-à-vis other tech-

niques. Moreover, they provide recommendations for future researchers inter-

ested in using this approach. Finally, they conclude that although this approach

poses unique challenges and drawbacks, when an unbiased sampling frame exists

or can be constructed, it allows researchers to inexpensively gather data with less

effort than other available approaches.

Surveying techniques—usually classified by the communication medium used, such

as face-to-face, telephone, mail, or electronic—rely on questioning individuals to

elicit particular information to look for patterns among facts, values, behaviors, and so

on to make generalizations about a population from which only some individuals are

surveyed. Over the years, the use of such techniques has been, by far, the most com-

mon method of data collection in several fields, and this is anticipated to remain such,

at least for the foreseeable future (Aaker, Kumar, & Day, 1995; Chadwick, Bahr, &

Albrecht, 1984; Malhotra, 1993; Synodinos & Brennan, 1988). Despite its well-

known inherent weaknesses relative to experimental methods, gathering data via sur-

veys has been more prevalent in management research arguably because of costs and

obstacles associated with carrying out experiments and because the basic locus of

many research questions has involved phenomena in the field. Simply put, if you want

to find out what managers are thinking, you need to ask them (Zikmund, 1994).

Albeit sketchy, the history of surveys and thus surveying techniques can be traced

back thousands of years (Erdos, 1983; Rossi, Wright, & Anderson, 1983). However,

until recently, mail questionnaires, field interviews, and telephone surveys were the

Authors’ Note: We wish to thank Monica Maciel Lopes and Melissa Foreman for their help during the

preparation of this manuscript. We also thank two anonymous reviewers for their suggestions.

Organizational Research Methods, Vol. 3 No. 1, January 2000 93-115© 2000 Sage Publications, Inc.

93

only convenient techniques to collect survey information. Gates and Jarboe (1987)

argue that developments in electronic technology, computer software, and environ-

mental forces that oppose traditional data collection techniques have contributed to

the change in data collection techniques today. Of these, computer technology has

been a fundamental force behind the growth of alternative survey techniques and has

led to improved data collection (Malhotra, 1993). Indeed, the emergence of this tech-

nology has affected not only data collection techniques but also has had a dramatic

impact on almost every phase of survey research, including instrument design, sam-

pling, field monitoring, coding and data editing, data capture, data cleaning, scale

index construction, database organization, database retrieval, data analysis, and docu-

mentation (Anderson & Gansender, 1995; Karweit & Meyers, 1983; Neal, 1989;

Saltzman, 1995).

It has been aptly acknowledged that currently there are probably as many surveying

techniques as there are different forms of communication technology (Aaker et al.,

1995). For example, computer-assisted personal interviewing (Baker, 1992; Couper &

Burt, 1994), computer-assisted telephone interviewing (Havice, 1990), fully auto-

mated telephone interviewing (Dacko, 1995), computer disk by mail (Higgins, Dim-

nik, & Greenwood, 1987; Saltzman, 1995), fax surveys (Vazzana & Bachmann,

1994), online World Wide Web (WWW) surveys, and focus groups (Gaiser, 1997) are

only a few of the new techniques that have evolved. This evolution is the result of rapid

developments in computer and communication technologies. The ever-increasing

preference for computer-mediated communication, the opening of the Internet to the

public, the introduction of WWW in 1989 at the European Particle Physics Laboratory

in Europe, and the low-cost dispersion of software and hardware are fundamental

forces that have shaped the viability of the e-mail survey technique (EST).

EST holds considerable promise to obliterate the time and geographical constraints

usually associated with surveys, facilitate interaction between surveyors and respon-

dents, and reduce cost, time, and data entry errors per response (Bachmann, Elfrink, &

Vazzana, 1996; Kiesler & Sproull, 1986; Mehta & Sivadas, 1995; L. Parker, 1992).

Nonetheless, despite this potential, our review of the literature revealed that research

on EST (a) is dispersed across the literature of almost two decades and several fields;

(b) has not been systematically evaluated, let alone integrated; (c) consists mostly of

empirical studies dealing with either response rates or quality of collected data or com-

mentaries that juxtapose the pros and cons; and (d) has not embraced replications and

lacks theoretical arguments and a conceptual framework. Indeed, to date, no one has

attempted to integrate and assess both the theoretical and practical concerns (cf.

Kiesler & Sproull, 1986; Kittleson, 1995; Oppermann, 1995; Schuldt & Totten, 1994).

Thus, it seems that EST is, like the weather, something about which everybody is talk-

ing but nobody is doing much about it.

We believe that this state of research on EST is unfortunate for several important

reasons. First, literally hundreds of organizations conduct e-mail or web-based sur-

veys for private and organizational consumers who in turn base their decisions on

these data. Second, on any given day, numerous researchers conduct surveys using

some conventional techniques, some of which could be done more efficiently and

effectively using EST. Third, although e-mail is the most widely used computer-

mediated communication medium, its full utility in sample surveying is not thor-

oughly assessed, let alone realized. Indeed, it took 2,500 years until basic postal serv-

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE 94

ices became available to most individuals after being established by King Cyrus of the

Persian Empire in the 6th century B.C. (Kochmer & Northwest, 1993, p. 47), but it took

only 50 years until the numbers of computers rose to 1 computer per 44 persons from

no more than dozens in the world (Perl & Whitley, 1995, p. 6). According to a recent

estimate, electronic messages sent in the United States went from 776 billion to 2.6

trillion from 1994 to 1997 and are projected to reach 6.6 trillion by 2000 (Gwyne &

Dikerson, 1997). Fourth, with the exception of earlier work by Kiesler and Sproull

(1986), there is a paucity of focused discussion on EST in the management literature.

Accordingly, using a three-dimensional framework focused on sampling issues,

nonsampling errors, and comparative performance, we review and integrate previous

research on EST and provide an assessment of the comparative trade-offs vis-à-vis

other techniques. Moreover, we provide recommendations for future researchers

interested in using this approach.

Although the basic notion for an electronic mail system has been around since the

1840s, e-mail has only emerged in the past three decades as a result of the convergence

of computer and communication technologies (Helliwell, 1986; Mortensen, 1985).

The use of e-mail was first started on the ARPAnet during the 1960s. At this early

stage, the limited access and primitive nature of the systems hampered widespread

usage, and it was not until local-area networks (LANs) were developed that e-mail

acquired its popularity.

E-mail is simply a combination of software, hardware, and communication tech-

nologies that allows a user to send and receive messages or documents to and from a

user or set of users. The Electronic Mail Association (EMA), a Washington-based

trade association, defines e-mail as “the generic term for the non-interactive commu-

nication of data, images or voice messages between a sender and designated recipi-

ent(s) by systems utilizing telecommunication links.” Although this definition encom-

passes technologies such as facsimile, telex, and communicating word processors, in

this article, e-mail refers to the transmission of text message and (in some advanced

systems) graphics, video, and sound over telephone lines or wireless technology from

computer to computer.

Within this context, EST can simply be defined as a computerized self-

administered questionnaire in which the researcher sends a questionnaire and the

respondents receive, complete, and return the questionnaire through e-mail systems

that bring together capabilities of both computers and telecommunication networks

(Rice, 1990). The researcher can send a separate e-mail with the survey embedded to

each respondent or multiple respondents, or the researcher can ask each respondent

to access a web site where the survey is housed. In this latter case, on completion, the

survey is submitted either as an e-mail to the researcher, or it can be downloaded to a

data file. Anyone with a computer, modem, and telephone line can use EST if he or

she has access to an online service, a commercial carrier, a LAN, or an Internet serv-

ice provider.

EST differs from WWW surveys that depend on the transmission of a questionnaire

over the Internet to a database located at the site of the study (Subramanian, McAfee,

& Getzinger, 1997) but rely on chance that somebody might come across the question-

95 ORGANIZATIONAL RESEARCH METHODS

naire, become interested in it, copy the document, complete it, and then return it (Swo-

boda, Muhlberger, Weitkunat, & Scheneeweib, 1997). Assuming no specialized con-

trols such as passwords are used, anyone who has access to the Internet could

potentially respond to such a survey. As such, although a WWW survey does not have

the sampling controllability as EST (Stanton, 1998), it has the potential, with protec-

tion, to provide greater anonymity. Furthermore, unlike EST, WWW surveys can

incorporate images, graphics, sound, and so on into the survey. Another advantage of a

WWW survey is that it is always present and available, whereas EST is inherently epi-

sodic (Stanton, 1998). EST, however, does not require the researcher to develop

appropriate data-screening methods and develop links that respondents should access

to participate in the survey.

EST also differs from surveys that are done through newsgroups or list servers,

although sending e-mail questionnaires through newsgroups and list servers is clearly

another form of EST. Like WWW surveys, these surveys suffer from low sampling

control as well as self-selection bias because individuals on newsgroups or list servers

are usually interested in particular issues. For example, a list server survey typically

relies on surveying individuals with a special interest in a certain topic (Stanton,

1998). The potential receivers are often unknown to the sender and are characterized

only by their interest in a particular subject (Batinic, 1997). There is virtually no con-

trol over individuals who are to complete the survey by the researcher (Swoboda et al.,

1997).

Once the target population is identified, determination of the sampling frame,

selection of a sampling procedure (probability vs. nonprobability), and computation

of the sample size are important sampling-related issues that must be addressed while

preparing for collecting data through EST. Although the latter two issues are rather

straightforward statistical topics, we will attend primarily to the availability or con-

struction of the sampling frame. The sampling frame can significantly reduce difficul-

ties involved in the sampling process because it determines the sampling control and

guides the direction of the inquiry in EST.

The sampling frame is a master listing of population members usually used to draw

a random sample from which data will be collected. Depending on the objective of the

research, a sampling frame can be a listing of all the managers who work at a company,

all the executives from Fortune 500 companies, and so forth. The quality of such a list

primarily determines sampling biases. The ideal sampling frame is one in which every

element of the population is not only represented but also only represented once.

Clearly, when one is interested in sampling an appreciable segment of the human

population, there will be problems with this ideal (Sudman, 1996; Tull & Hawkins,

1993). With respect to telephone and mail survey techniques, problems such as mail-

ing lists being out of date or incomplete and phone directories not including unlisted

numbers have been addressed. However, mostly because of lack of reliable documen-

tation on addresses and profiles of e-mail users among different segments of the soci-

ety, problems associated with sampling frames of EST have not been documented yet.


To our best knowledge, there is no good frame that lists individuals or households

using or even having access to e-mail, although it has been reported that the Internet’s

Network Information Center is working to produce a master directory of all users

called InterNIC (T. Parker, 1995). We also do not know of any private firm that can

currently offer unbiased frames for many populations. Indeed, there are many list bro-

kers, list servers, and sources of opt-in lists on the Internet, the lists that are developed

from people who have agreed to receive unsolicited e-mail. Yet, it has been suggested

that commercial lists tend to be seriously flawed (Comley, 1996). Although it might be

possible to create a good EST sampling frame by using e-mail addresses from online

service providers or e-mail carriers, these companies observe privacy laws and poli-

cies, so this is not a viable option. Finally, even though there are several utilities such

as Finger, Whois, and Netfind, to collect the Internet e-mail addresses, it has been

claimed that only 1% or 2% of all the Internet users can be located with one of these

methods (T. Parker, 1995). Accordingly, obtaining or constructing an unbiased or at

least a useable sampling frame that allows probability sampling is currently the most

serious challenge that EST imposes on researchers. Indeed, whether EST can success-

fully use probability sampling for general populations has not yet been established.

On the other hand, even when available, many if not all e-mail frames should be

used cautiously because they usually lack universal coverage of the population. As a

rule, such claims as those echoed in the popular media on availability or accessibility

of e-mail should not be considered as a surrogate measure of feasibility. In making a

case, such aggregate numbers are of limited use because the availability of e-mail does

not guarantee acceptance, usage, or compatibility (Kerr & Hiltz, 1982; Komsky,

1991). Although researchers have generally been able to assume that people receive

their mail at their postal address, they cannot simply assume that all e-mail addressees

are active.

Moreover, although some of the problems in constructing e-mail frames pertain to

self-administration in general (e.g., literacy and blindness), others may only relate to

the ability to have and use a computer. The systematic exclusion associated with

e-mail frames is severe, particularly because of e-mail’s relation to purchasing power.

Research on computer usage reveals that computer users still share similar demo-

graphic characteristics of being young, well educated, and above average in income

(Oppermann, 1995). Likewise, Couper and Rowe (1996) found that less educated,

older respondents and those with less computer experience were less likely to com-

plete a self-administered component of a computer-assisted personal interview survey

on self-images, suggesting that the use of the computer may add additional constraints

on the willingness or ability of respondents to complete an e-mail questionnaire.

Clearly then, whether researchers can currently reach a representative sample

through EST primarily hinges on the population under investigation. For example, if

the investigation involves low-income households, minorities, or elderly populations,

even simple random sampling attempts will be flawed because of high noncoverage

error (Anderson & Gansender, 1995; Dillman, 1991). On the other hand, EST can

prove quite beneficial for obtaining opinions related to new software. Likewise,

because most large firms and their managerial/professional employees have access to

e-mail, sample surveys of these populations are possible. In any case, using a stratified

sampling approach rather than a random sample should lessen the degree of potential

noncoverage error (Oppermann, 1995). However, the researcher should keep in mind


that almost all hypothesis testing and estimation procedures assume simple random

samples, not stratified samples.

Nonsampling errors, which are often viewed more important than sampling errors,

are all the other errors in a survey except those due to sampling method and sample size

(Sudman, 1996; Tull & Hawkins, 1993). More specifically, they include coverage

error—which has been discussed—nonresponse, and measurement error (Lavrakas,

1996).

Nonresponse errors. A high number of nonresponses raise the question of whether

those who responded to the survey are different from those who did not. Even in the

absence of sampling biases, if nonresponses are not randomly distributed, then the

data generated by EST will be biased because careful attempts at sample randomiza-

tion have been eroded. In turn, such biased data severely influence the validity of the

research and often result in invalid inferences (Gilbert, Longmate, & Branch, 1992;

Dillman, 1991; Dillon, Madden, & Firtle, 1987). Nonresponses, even when random,

may reduce what was an adequate sample to an inadequate one, forcing the researcher

to either survey additional respondents or to find a remedy for these missing responses

through postsurvey estimates (Hair, Anderson, Tatham, & Black, 1995). Nonetheless,

increasing the response rate—as opposed to postsurvey adjustments, such as weight-

ing cases by estimated probabilities of cooperation and known population quantities,

imputation, and selection bias models that can work under certain assumptions to a

limited extent (e.g., Hair et al., 1995; Kalton, 1983)—is clearly the safest strategy to

reduce nonsampling errors.

Overall, nonresponse errors in EST can generally be attributed to noncontacts (i.e.,

unreachables and refusals). It is promising to note that EST has been used as an effec-

tive means of gathering data in several academic and institutional settings (Anderson &

Gansender, 1995), especially when one takes into account that most responses were

attained without the use of any response inducement technique.

Researchers using EST have reported response rates ranging from 19.3% to 76%.

For example, Kiesler and Sproull (1986), on examining the response rate associated

with EST and a postal survey, found a higher response rate for the paper survey (75% vs.

67%). When comparing EST with face-to-face interviews in a Fortune 500 manufac-

turing company, Sproull (1986) found a participation rate of 73% for EST versus 87%

for interviews. In a survey of a major corporation’s overseas employees, L. Parker

(1992) reported that the response rate associated with EST (68%) was significantly

higher than when mail pouches (38%) were used. Although Schuldt and Totten (1994)

reported a response rate of 56.5% for a mailed survey and 19.3% for EST, Kittleson

(1995) obtained a response rate of 28.1% for EST and 76.5% for a postcard survey.

Anderson and Gansender (1995), employing a survey to assess how and why people

used a network system, obtained a response rate of 76% from 488 Free-Net users of a

metropolitan area. Walsh, Kiesler, Sproull, and Hesse (1992) attained a response rate

of 76% from a 93-item online survey of 300 science-net subscribers. Finally, in

another study involving business school deans and division chairpersons on the use of

total quality management, Bachmann, Elfrink, and Vazzana (1996) had a response rate

of 65.6% for the mail questionnaire and 52.5% for EST.


Although further research is need on factors causing response rate differences

across studies, these findings overall indicate that EST has been used as an effective

means of gathering data in terms of response rates. They also indicate that EST tends to

have a lower response rate compared to mail surveys, thereby pointing to the need for

more research on response inducement techniques in the EST context. However, aug-

menting responses to EST through some incentives and procedures virtually has not

been explored. Even fewer studies have investigated the influence of such tactics and

incentives on response speed and response content. Although the researcher has no

control over the unreachables, refusals stemming from such factors as survey design

can be influenced as the research on mail survey suggests (Linsky, 1975; Yammarino,

Skinner, & Childers, 1991). In the Recommendations section of this article, we further

touch on this issue.

Measurement errors. Measurement error is simply the deviation between the “true”

and the observed responses (Dillman, 1991). Many types of errors stemming primarily

from the data collection method used, such as those committed during the transforma-

tion of the data, are always at work in a survey. Broadly speaking, there are three

sources of measurement error due to the survey instrument, the respondent, and/or the

data collection technique. With respect to EST, measurement errors due to the survey

instrument generally occur at the presurvey scale development stage. In this respect,

EST does not differ from other surveying techniques in that it also requires the

researcher to do everything that is needed in terms of developing a reliable and valid

scale. Given the newness of EST, we were able to find only one study that detailed how

the scale for a study using EST was developed (Clayton, Applebee, & Pascoe, 1996).

With respect to respondent-based errors, some researchers have examined

response content of computerized data collection techniques in general and EST in

particular. Tourangeau and Smith (1996) noted that computer-assisted, self-

administered surveys produced similar outcomes to the more conventional self-

administered techniques. They further claimed that computerization by itself had little

influence on the response quality and that better data quality often associated with the

computer-administered questionnaires may be the result of the self-administration.

However, in a comprehensive review on computerized data collection techniques,

Leeuw, Hox, and Snijkers (1995) contended that in general, computerized techniques

such as computer-assisted personal interviewing (CAPI) and computer-assisted self-

interviewing (CASI) have a positive effect on data quality. In the case of EST, these

findings are largely consistent with the findings reported on response quantity, but

they are inconclusive with respect to response quality because how responses are

influenced by e-mail communication itself has not been conclusively demonstrated.

Some researchers have asserted that EST generally conveys little social informa-

tion, so respondents experience less evaluation anxiety than when they respond using

other survey modalities (Kiesler & Sproull, 1986; Kiesler, Zubrow, & Moses, 1985;

Sproull, 1986). Kiesler (1989) and Sproull (1986) suggest that because e-mail tends to

reduce social concerns and constraints on individuals, EST respondents are less con-

cerned about reporting negative and socially inappropriate things about themselves.

Couper and Rowe (1996) found that those who completed a self-administered com-

puter interview reported a more positive self-image than those who had the inter-

viewer help, after controlling for respondent characteristics related to self-image.

Ayidiya and McClendon (1990) noted that EST might influence the acquiescence of


respondents, thereby lessening some respondents’ propensity to agree with survey

statements more often than they would do with a pencil-and-paper survey.

In another study, Sproull (1986) observed that questionnaire data were slightly

more complete when using conventional mail than when e-mail was used. However,

when comparing these methods, it was found that although there were no differences

in the nature of answers provided by the participants, the e-mail survey elicited more

extreme responses. Perhaps this is because, as Kiesler (1989) has suggested, e-mail

communication loosens social concerns and constraints on people, so that they are less

concerned about saying negative things and/or revealing socially inappropriate

beliefs.

Indeed, in a meta-analysis of self-disclosure on computer forms in general, Weis-

band and Kiesler (1996) found that across 39 studies using 100 measures, computer

administration was associated with increased self-disclosure compared to face-to-face

interviews. The researchers speculated that this finding might be because working on a

computer creates a sense of privacy. Corman (1990) directly attempted to validate data

generated by a computerized survey by comparing them with postal survey data. To do

this, data from two separate groups were compared in three different ways, including

test-retest reliability, criterion validity, and internal consistency. The results indicated

that the computerized survey method produced considerably higher criterion validity

and slightly higher test-retest reliability than did the postal survey. Corman attributed

these findings to the novelty of the computerized data collection approach in that com-

puter respondents may have taken greater care in filling out the survey.

With respect to measurement errors introduced by the data collection technique

itself, several researchers have looked at how EST compares to the postal survey tech-

nique in terms of item completeness and responses to open-ended questions. Liefeld

(1988) compared the response effects of a computer-administered questionnaire to

self-completion and personal interview techniques. Liefeld found that with the excep-

tion of multiresponse/knowledge-type questions, there was little difference in

response patterns among techniques. The computer-assisted technique, however, pro-

duced higher means for most of the items. Bachmann et al. (1996) also found that there

were no significant differences in the responses and respondents’ tendency to leave an

item blank or to comment on questions between individuals receiving a mail or an

e-mail questionnaire. Yet, the e-mail respondents showed a greater willingness to

respond to open-ended questions (21.9% vs. 4.8%). Kiesler and Sproull (1986) found

similar responses between a paper and an electronic survey but also reported greater

item completeness resulting for the electronic survey. Finally, Schaefer and Dillman

(1998) found that EST generated greater item completion and lengthier responses to

open-ended questions. In turn, all this suggests that EST might be particularly useful to

conduct surveys involving open-ended questions.

On the other hand, some have expressed concern that self-administered question-

naires can suffer from sequence biasing because respondents are able to see the whole

questionnaire and may consider questions together rather than each question individu-

ally (Churchill, 1995, p. 371). Hence, responses to an earlier question can prime par-

ticular beliefs and make them more accessible, serve as a standard of comparison for

subsequent items, or be a source for consistency pressure (Lockhart & Russo, 1996).

Although sequencing bias is a concern for conventional paper surveys, e-mail ques-

tionnaires may be sent in such a way that the computer displays each question


exactly as wished and does not display all the questions until previous ones have

been completed.

Table 1 displays some other specific possible sources of nonsampling errors and

some techniques for reducing or controlling them when using EST. Table 1 is based on

the extant research on self-administered questionnaires in general. Thus, most of our

suggestions are not unique to EST because they represent general guidelines for iden-

tifying or controlling some nonsampling errors.

The recent proliferation of data collection techniques has made it important that

researchers have a comparative knowledge of these techniques to make an appropriate

decision. For example, although an electronic survey may generate a lower response

rate compared to a postal survey, the choice of which one to use might also depend on

such issues as survey costs, speed, convenience, and the like. Several researchers have

indicated that the assessment of any data collection technique should take into account

its comparative performance as well (Lockhart & Russo, 1996, p. 125). Salant and

Dillman (1994) noted that “no single method can be judged superior to the others in the

abstract. Instead each should be evaluated in terms of a specific study topic and popu-

lation, as well as budget, staff, and time constraints” (p. 35). More specifically, Kiesler

and Sproull (1986) aptly suggest comparing EST with other available alternatives as a

good first step. Accordingly, Table 2 compares EST with personal interviewing, tele-

phone interviewing, and the mail questionnaire technique along several dimensions

considered to be important determinants of the choice of a data collection technique

(cf. Dillman, 1978, 1991; Dillon et al., 1987; Erdos, 1983; Markus, 1994; Watson,

1998). These determinants include sampling issues, cost/efficiency and convenience,

information richness, respondent issues, response outcomes, and future prospects in

terms of usage.

Given that we have already addressed sampling issues, we will first turn to issues of

cost, efficiency, and convenience. EST has the potential of radically changing the eco-

nomics of conducting surveys. With the file transfer capability of computers, EST

does not require usage of paper at any stage, thereby avoiding the costs associated with

the manual entry of raw data or electronic scanning. With EST, according to one esti-

mate, the marginal cost of storage, communication, and dissemination of a 30-page

document can be less than a penny (Kambil, 1995). In many cases, a hard copy of ques-

tionnaires may not be necessary, which in turn eliminates the need to print labels, type

addresses, purchase envelopes, and so on. Furthermore, although the costs of the other

techniques tend to be proportional to the size of the sample, the cost associated with

adding additional respondents in EST is practically zero. The primary costs of EST

include assembling and checking the e-mail list(s), creating or buying software and

supporting databases, and accessing e-mail. When these are available, the cost of EST

is trivial to the researcher. In any case, the marginal costs of collecting and communi-

cating data through EST are much lower than costs of interviewing, telephoning, and

sending questionnaires through postal services (Mehta & Sivadas, 1995).

With respect to speed, sending questionnaires out and receiving them via EST are

definitely very fast. An e-mail questionnaire can be sent to one thousand people as eas-

ily as to one person automatically, and all potential respondents immediately receive


(text continued on p. 104)

Tabl

e1

Som

eN

onsa

mpl

ing

Err

ors

inE

ST

and

Tech

niqu

esfo

rH

andl

ing

The

m

Type

Def

initi

onS

ome

Pos

sibl

eTe

chni

ques

for

Han

dlin

g

Non

resp

onse

Failu

reto

obta

inin

form

atio

nfr

omso

me

elem

ents

ofth

esa

mpl

e.U

nrea

chab

les

Des

igna

ted

resp

onde

ntca

nnot

bere

ache

dvi

aE

ST.

1.C

heck

the

e-m

aila

ddre

ssfo

rac

cura

cy.

2.C

heck

for

tem

pora

rylo

cala

ndno

nloc

alsy

stem

wid

ee-

mai

lpro

blem

s.R

efus

als

Res

pond

ents

dono

tres

pond

toth

equ

estio

nnai

re.

1.U

sepr

ior

e-m

ailn

otifi

catio

ns.

2.A

ttem

ptto

conv

ince

the

resp

onde

ntof

the

valu

eof

the

rese

arch

and

his

orhe

rpa

rtic

ipat

ion.

3.E

nsur

ean

onym

ityan

dco

nfid

entia

lity.

4.In

crea

secr

edib

ility

thro

ugh

spon

sors

hip

man

ipul

atio

n.5.

Offe

rso

me

ince

ntiv

essu

chas

gifts

orm

oney

tom

otiv

ate.

6.S

hort

enth

equ

estio

nnai

rew

hen

poss

ible

.7.

Sen

din

divi

dual

e-m

ailq

uest

ionn

aire

sra

ther

than

forw

ardi

ngor

usin

gch

ain

send

ing.

8.M

ake

the

ques

tionn

aire

appe

alin

gth

roug

hvi

sual

aids

.9.

Use

ane-

mai

lfol

low

-up.

10.

Ove

rsam

plin

g.R

espo

nden

terr

ors

Alth

ough

resp

onde

nts

part

icip

ate

inth

est

udy,

thei

rre

spon

ses

are

notc

ompl

ete

orin

corr

ect.

Inte

ntio

nal

Res

pond

ents

fail

tote

llth

etr

uth

ordo

notp

rovi

de1.

Em

phas

ize

anon

ymity

.re

spon

dent

erro

rsre

spon

ses

toso

me

item

s.2.

Use

valid

atio

nch

ecks

.3.

Use

ath

ird-p

erso

nte

chni

que

whe

npo

ssib

le.

Uni

nten

tiona

lR

espo

nden

tsfa

ilto

prov

ide

thei

rre

alth

inki

ngbe

caus

e1.

Scr

een

surv

eyqu

estio

nsbe

fore

usin

g.re

spon

dent

erro

rsth

eym

isun

ders

tand

the

ques

tions

,or

they

gues

s.2.

Use

valid

ated

scal

e.3.

Pro

vide

good

ques

tionn

aire

inst

ruct

ions

and,

whe

npo

ssib

le,u

seex

ampl

es.

4.U

sere

vers

als

ofsc

ale

end

poin

tsor

ques

tion

stat

emen

ts.

5.U

sevi

sual

aids

whe

npo

ssib

le.

Exc

eptio

nsR

espo

nden

tsex

hibi

tape

rsis

tent

tend

ency

tore

spon

d1.

Rec

heck

the

ques

tions

’wor

ding

.fa

vora

bly

orun

favo

rabl

y,ha

veno

opin

ion,

orpr

ovid

e2.

Exc

lude

inco

nsis

tent

resp

onse

s.in

cons

iste

ntre

spon

ses.

3.Tr

eats

ome

resp

onde

nts

asou

tlier

s.

NO

TE

:ES

T=

e-m

ails

urve

yte

chni

que.

102

Tabl

e2

ES

T,M

ail,

Per

sona

l,an

dTe

leph

one

Dat

aC

olle

ctio

nTe

chni

ques

Com

pare

d

ES

TM

ailQ

uest

ionn

aire

Per

sona

lInt

ervi

ewTe

leph

one

Inte

rvie

w

Sam

plin

gIs

sues

Sam

plin

gfr

ames

for

man

yS

ampl

ing

fram

efo

rm

any

Sam

plin

gfr

ames

are

usua

llyS

ampl

ing

fram

esfo

rso

me

popu

latio

nsdo

note

xist

and

popu

latio

nsex

ista

ndar

eea

sily

obta

ined

and

cons

truc

ted.

popu

latio

nsal

read

yex

ista

ndar

eus

ually

diffi

cult

toob

tain

usua

llyea

syto

obta

inan

dH

igh

pote

ntia

lfor

sam

plin

gar

eus

ually

easy

toob

tain

and

cons

truc

t.Li

mite

dpo

ssib

ility

cons

truc

t.Li

mite

dpo

ssib

ility

cont

rol.

and

cons

truc

t.H

igh

sam

plin

gof

sam

plin

gco

ntro

l.of

sam

plin

gco

ntro

l.co

ntro

l.

Cos

teffi

cien

cyLe

aste

xpen

sive

and

mos

teffi

cien

t.Lo

wco

st/m

oder

atel

yef

ficie

nt.

Hig

hco

stan

dle

aste

ffici

ent.

Hig

hest

cost

and

leas

teffi

cien

t.

Info

rmat

ion

richn

ess

Low

tran

smis

sion

ofno

nver

bal

Bas

edon

the

sam

efo

urcr

iteria

Bas

edon

the

sam

efo

urcr

iteria

Bas

edon

the

sam

efo

urcr

iteria

cues

,con

veyi

nglo

wse

nse

ofof

med

iaric

hnes

s,its

richn

ess

ofm

ediu

mric

hnes

s,it

isth

eof

med

ium

richn

ess,

itis

the

pers

onal

izat

ion,

timel

yfe

edba

ck,

isth

elo

wes

t.ric

hest

ofth

efo

urte

chni

ques

.se

cond

riche

stof

the

four

and

tran

smis

sion

ofm

ediu

mte

chni

ques

.va

ried

lang

uage

.

Res

pond

enti

ssue

sH

igh

resp

onde

ntco

nven

ienc

e—M

axim

umre

spon

dent

Low

resp

onde

ntco

nven

ienc

e—Lo

wre

spon

dent

conv

enie

nce—

time

and

disc

retio

nto

resp

ond.

conv

enie

nce—

time

and

notim

ean

ddi

scre

tion

tore

spon

d.no

time

and

disc

retio

nto

Som

ean

onym

ityis

poss

ible

.di

scre

tion

tore

spon

d.F

ull

resp

ond.

Req

uire

slit

erac

yan

dco

mpu

ter

anon

ymity

poss

ible

.Req

uire

ssk

ills;

com

patib

ility

ofe-

mai

llit

erac

y.pa

ckag

esco

uld

bea

prob

lem

.

103

Res

pons

eou

tcom

esM

ediu

mto

high

.No

inte

rvie

wer

Med

ium

tolo

w.N

oin

terv

iew

erH

ighe

st.P

ossi

bilit

yof

inte

rvie

wer

Med

ium

tohi

gh.P

ossi

bilit

yof

dist

ortio

n,le

ssso

cial

lyde

sira

ble,

dist

ortio

n,le

ssso

cial

lydi

stor

tion

and

soci

alde

sira

bilit

y.in

terv

iew

erdi

stor

tion

and

thou

ghtfu

lres

pons

es,l

owde

sira

ble,

thou

ghtfu

lres

pons

es,

Hig

hlik

elih

ood

that

cont

amin

atio

nso

cial

desi

rabi

lity.

Hig

hlik

eli-

invo

lunt

ary

erro

r,an

dse

quen

celo

win

volu

ntar

yer

ror,

and

from

othe

rsav

oide

d.Lo

w-

hood

that

cont

amin

atio

nfr

ombi

asin

gpo

ssib

ility

.App

ropr

iate

sequ

ence

bias

ing

poss

ibili

ty.

sequ

ence

bias

ing

poss

ibili

ty.

othe

rsav

oide

d.Lo

w-

sequ

ence

for

elic

iting

sens

itive

resp

onse

s.A

ppro

pria

tefo

rel

iciti

ngC

ontr

olov

erits

spee

d.N

otbi

asin

gpo

ssib

ility

.Con

trol

over

No

cont

rolo

ver

resp

onse

spee

d.se

nsiti

vere

spon

ses.

No

cont

rol

appr

opria

tefo

rel

iciti

ngse

nsiti

veits

spee

d.Li

mite

dap

plic

a-bi

lity

Ver

yea

syto

proc

ess

and

anal

yze.

over

itssp

eed.

Eas

yto

data

.Ofte

ndi

fficu

ltto

proc

ess.

toel

icit

sens

itive

data

.Of-

ten

Sel

f-se

lect

ion

bias

poss

ibili

ty.

proc

ess

and

anal

yze.

Sel

f-Lo

wite

mno

nres

pons

e.ea

syto

proc

ess.

Low

item

Less

pron

eto

field

erro

rs.

sele

ctio

nbi

aspo

ssib

ility

.N

onre

spon

se.

Fut

ure

pros

pect

sH

igh

likel

ihoo

dof

wid

epo

pula

rity.

Dec

reas

ing

popu

larit

y,H

igh

likel

ihoo

dth

atits

usag

ew

illH

igh

likel

ihoo

dth

atits

us-

age

Priv

acy

conc

erns

may

limit

itspr

opor

tiona

teto

incr

ease

inno

tbe

influ

ence

dby

incr

easi

ngw

illre

mai

nhi

ghin

the

near

usag

e.E

ST

and

tele

phon

ein

terv

iew

ing.

use

ofe-

mai

lsur

veys

.How

ever

,fu

ture

aste

leph

one

com

-m

uni-

incr

easi

ngus

eof

com

pute

r-ca

tion

gets

chea

per.

Am

ong

assi

sted

inte

rvie

win

g,te

leph

one

high

e-m

ailu

ser

popu

la-

tions

,in

terv

iew

ing,

and

give

nvi

deo-

itsus

age

mig

htde

crea

se.

conf

eren

cing

capa

bilit

yof

com

pute

rs,i

tsus

age

amon

gaf

fluen

tpop

ulat

ions

mig

htde

crea

se.

NO

TE

:ES

T=

e-m

ails

urve

yte

chni

que.

104

the questionnaire regardless of their location (Kiesler & Sproull, 1986). Likewise,

responses can flow back just as rapidly when respondents check their e-mail daily.

None of the existing survey techniques, including facsimiles, can provide researchers

with such speed in reaching specified individuals. Thus, EST promises to provide data

in a timely manner when a quick answer is being sought. Indeed, as Mehta and Sivadas

(1995) have suggested, EST generates fast data not only because e-mail is a fast com-

munication medium but also because individuals are likely to respond more quickly to

an e-mail, whereas in comparison, a mail questionnaire may remain on an individual’s

desk for a long time.

Likewise, EST is one of the most convenient data collection techniques. Once the

survey instrument is developed, it can be e-mailed easily. EST saves all the time that

regular postal survey requires for photocopying questionnaires, stuffing envelops, and

addressing outgoing mail. Because a copy of all outgoing e-mail can be saved in an

electronic mailbox, EST also makes repeated communications with respondents, such

as sending follow-up questionnaires, extremely easy. Given that EST allows the

researcher to know in a moment if the message has been received—and even when it is

opened—identification, elimination, and replacement of unreachable respondents are

easily accomplished. EST can also reduce many field and administration errors, such

as deciphering respondents’ handwriting and allowing researchers to encode data

without transcribing from paper.

According to information richness theory (Daft & Lengel, 1984; Daft, Lengel, &

Trevino, 1987), computer-mediated communications, such as electronic mail, are less

rich in information-carrying capacity than face-to-face communication. Within this

perspective, face-to-face interaction is seen as the richest medium, followed by tele-

phone, electronic mail, letters, and memos. In effect, e-mail offers limited interactivity

and informational cues compared to face-to-face interactions. Indeed, compared to

other surveying techniques such as personal and telephone interviewing, EST

involves low transmission of nonverbal cues, varied language, timely feedback, and

low sense of personalization.

With EST, lack of complete anonymity is also a concern. Truly anonymous

responses are not possible with EST. When a respondent returns a questionnaire using

the reply function in an e-mail package, his or her e-mail address, including the name

and affiliation, is automatically conveyed to the surveyor. This lack of anonymity

might in turn affect response rates as well as response content in EST. Moreover, if

respondents complete the survey at their place of employment, it is possible that an

electronic trail will remain and that their responses could be uncovered, which,

depending on the nature of the survey questions, could raise confidentiality issues as

well. Previous research has indicated that respondents’beliefs about anonymity affect

responses to computer-based surveys (Kantor, 1991). There is also a wide recognition

among researchers that whether anonymity is provided affects responses in mail sur-

veys (Albaum, 1987). However, if anonymity or confidentiality is a major concern, we

suggest one of two approaches: First, use a web-based survey because when respon-

dents submit their answers back to the researcher, their identifying information is not

automatically conveyed, and second, respondents could be directed to go through a

free Internet e-mail account such as “Hotmail” and, if necessary, use a fictitious name.

With respect to our second suggestion, note that it is extremely difficult, if not impossi-


ble, for an outsider to obtain the identity of a respondent, given the privacy assurance

policies that such providers offer. Indeed, most providers indicate that the only excep-

tion to such a policy is in the case of an alleged crime. Hence, we believe that this

approach would provide a high degree of confidentiality. However, having said that,

irrespective of the actual degree of confidentiality achievable by such approaches,

EST researchers should be aware that as long as respondents perceive that complete

confidentiality is necessary and could be compromised, response rates are likely to

suffer.

Noncompatibility among e-mail packages is another important disadvantage of

EST compared to the other three techniques. E-mail is still not a standardized medium,

despite the growing demand of users for standardization. For instance, LAN e-mail

packages are different not only from one another but also from the Internet e-mail.

Although most online service providers and LAN-based e-mail systems permit trans-

fer of binary files, the Internet still uses an ASCII format. Nonstandardization among

various e-mail systems causes discrepancies between the form of questionnaire sent

and that received by respondents (Oppermann, 1995). Relatedly, researchers should

also be concerned with such issues as free hard disk space (either in their or their serv-

ice provider’s hard drive), e-mail bandwidth, and server capacity while using EST. For

instance, a large number of responses could create problems because returned e-mail

takes up a lot of space on a system. It is essential to download responses in a timely

manner while using EST for full-scale surveys.

Finally, with respect to the future prospects of EST, Bloom, Milne, and Adler

(1995) have noted that although new information technologies increase efficiency and

effectiveness of data collection, their haphazard use can lead to some legal difficulties.

For example, although legislation concerning privacy of e-mail communication is still

in its infancy, such legislation would effectively destroy the use of EST in many con-

texts. As e-mail addresses are considered to be more personal than mail addresses,

sending unsolicited e-mail questionnaires might be considered an intrusion on a per-

son’s seclusion or solitude or into his or her private affairs (Dyson, 1994). Although

such a case has yet to be made in the courts to our knowledge, the use of e-mail for

mass mailing, known as “spamming,” has been very much debated. A popular online

service was sued in three states for deceptive advertising because some of its cus-

tomers were using the system for mass mailing instead of personal messaging (Shan-

non & Rosenthal, 1993).

The e-mail overload that many individuals increasingly face is likely to be another

important disadvantage of EST (Clayton et al., 1996). Some researchers have indi-

cated that the increasing e-mail overload can cause individuals to feel overwhelmed

(Garton & Wellman, 1995). It is likely that the more individuals receive e-mail, the less

likely they are to spend time responding to EST. Although individuals do not have

many clues for judging the importance of e-mail they receive, it is very easy for them to

sort out and delete e-mail that they are not interested in. In turn, this makes it particu-

larly important that researchers successfully employ some incentives and response

inducement strategies to make respondents complete the survey.

In sum, researchers have to make a number of trade-offs when they decide to use

EST. Using EST requires harmony among a variety of resources, including human,

hardware, and software. It is thus essential that the decision to use EST takes into

account all relevant issues, including sampling issues, nonsampling errors, and com-


parative performance that are discussed throughout this article. We next provide

some practical recommendations as to how EST can be used productively.

We suggest that researchers first go through relevant directories and then check the

following sources: The Usenet Addressees Database, Knowbot Information Server

(KIS), GOPHER, WAIS, The Usenet Newsgroup, and Netfind. Many of these sources

can be used to locate individuals’ e-mail addresses or names. For example, Mehta and

Sivadas (1995) wrote a program that collected e-mail addresses and signatures of the

people who posted articles on newsgroups; then, based on confirmation, they deter-

mined mailing addresses of individuals. Put simply, because some Internet users

already have organized themselves into mailing lists and discussion groups according

to their interests, it is possible to use these lists or combine several such lists to con-

struct specific sample frames. The directories seem to be especially beneficial if the

population of interest is academic staff because a growing number of institutions are

putting their staff and student directories online in publicly accessible formats and are

being incorporated into Gopher and WAIS (Kochmer & Northwest, 1993, p. 53).

It is also possible that the researcher can develop sampling frames using more con-

ventional sources such as mailing lists and phone directories and then use these

sources to determine e-mail addresses of individuals through several Internet search

engines or the aforementioned sources. In fact, some of the sources, such as Finger and

KIS, provide additional information such as telephone number, postal address, and so

on of individuals, thereby allowing the information obtained from the conventional

sources to be double-checked.

Based on our experience with e-mail surveys and the extant literature, we recom-

mend the following broad approaches. First, because many e-mail users have strong

concerns about the use of their e-mail boxes and consider them to be more private than

their mail addresses, it may prove beneficial to notify sample members about the

incoming e-mail questionnaire through an e-mail or postal prior notification (Emery,

1995, p. 344). The prior notification should not only ask for permission but also let the

respondent know the purpose of the survey, why their involvement is important, how

responses will be used, the sponsor of the survey, person(s) to contact for questions,

expected date of the survey, and a statement indicating the strict confidentiality of the

respondent’s e-mail address and response. Yu and Cooper (1983) suggest that a prior

notification should simultaneously include the following: (a) a social utility appeal

that emphasizes the worthiness of the survey, (b) an egoistic appeal that stresses the

respondent’s place and importance in completing the survey, and (c) an appeal to help

the researcher in completing an important project. Given concerns over anonymity in


EST, it is also important that the cover letter assure respondents that their responses

will be held confidentially and mention some possible steps that will be taken toward

this goal. For example, the researcher may state that screen headers will be deleted

once the responses are received (Goree & Marszalek, 1995). The researcher could also

offer some options for responding anonymously such as placing the questionnaire on

the WWW or mentioning the possibility that the respondent could send the completed

questionnaire through regular mail.

Second, questionnaire layout and design issues should also be taken into account.

EST should be accompanied by very clear and simple instructions, such as how to

reply, that will not consume much of the respondents’ time. In particular, “extra” fea-

tures that would minimize questionnaire completion time and maximize respondent

convenience should be pursued. For example, scrolling, jump screen, quitting, no

automatic next, no keyboard responses, help screens, and a progress thermometer

indicating completed percentage of the questionnaire were incorporated and success-

fully used by Beebe, Mika, Harrison, Anderson, and Fulkerson (1997). Like many

other researchers (e.g., Johnston & Walton, 1995), we also believe that whenever pos-

sible, researchers should use simple graphics-animated questionnaires because many

people’s perceptions of computers are similar to that of TV rather than postal mail. But

we caution that such devices consume enormous amounts of memory and make open-

ing such messages time-consuming and frustrating if the individual is on an older

modem. Graphics, sounds, and special formatting of the questionnaire may not trans-

late across various e-mail software packages. To solve such problems, the researcher

could check the major e-mail systems used by respondents and make sure that the for-

matting and appearance of the questionnaire remain the same after transmission (Tse,

1998). However, unless the researcher knows the capabilities of the e-mail systems of

the people included in the survey, we suggest that it is best to keep the survey as simple

and short as possible. As an aside, it should be remembered that respondents who use

commercial online services effectively incur some cost in sending and receiving mes-

sages.

Only after incorporating such approaches should the researcher attempt to manipu-

late some incentives and factors to increase responses to EST without eschewing that

different populations may react differently to the factors, which the large body of

research on mail survey strongly points out (Childers, Pride, & Ferrell, 1980; Jobber &

Sanderson; 1985; Kaldenberg, 1994). A plethora of studies have been undertaken to

identify factors that might potentially influence responses to a mail survey, including

monetary offerings, lottery tickets, contributions to a charity, an offer of survey

results, cover letter, personalization, anonymity, topical interest, sponsorship, ques-

tionnaire design, prior notification, follow-up, humor, type of mailing, and deadline.

Excellent reviews of this body of research have been written by Church (1993); Fox,

Crask, and Kim (1988); Heberlin and Baumgartner (1978); Jobber (1986); Linsky

(1975); Veiga (1984); Yammarino et al. (1991); and Yu and Cooper (1983). Although

EST may resemble a postal survey and share some characteristics of it, not all of these

response inducement techniques are transferable to the EST context because EST has

its own unique features. For example, it is impossible to attach a monetary incentive,

such as a dollar bill, to an electronic survey or to attach a nonmonetary incentive such

as a pen.


In a recent review of the literature on factors inducing responses to mail surveys,

Roth and BeVier (1998) concluded that prior notification, follow-up, monetary incen-

tives, personalization, and salience of the issues investigated have consistently been

found to positively influence response rates. Although not conclusive, a few recent

studies indicate that responses to EST can also be increased through these strategies

(e.g., Kittleson, 1997; Schaefer & Dillman, 1998). In addition to these factors, we also

expect that sponsorship might positively influence responses to EST. For example,

alumni may respond to a university or business school–sponsored surveys more read-

ily because of psychological indebtedness (Paxson, 1995). However, there is still lack

of focused research on determining how strategies such as sponsorship, the opportu-

nity to complete the survey through regular mail or the WWW, summary of the

research, donations to a charity, purchase credits, and discounts affect response rates

and response content in EST.

Although we have primarily focused on the use of EST as an alternative, independ-

ent surveying technique, EST can also be used in combination with some other tech-

niques or at different stages of a research project. In fact, it is sometimes desirable to

combine several techniques, thereby offsetting the strengths and limitations of any

single technique (Aaker et al., 1995; Lockhart & Russo, 1996). In addition, using EST

with some other techniques such as postal surveys can allow experimentation with

much more diverse populations, not only with populations having nearly universal

coverage. This strategy can reduce coverage error that is usually associated with EST

as well (Schaefer & Dillman, 1998).

EST can be used in combination with almost any data collection technique, includ-

ing telephone interviews, personal interviews, postal surveys, or the other Internet-

based surveys, as well as to send prior notifications and follow-ups. Schaefer and Dill-

man (1998) suggested that because of its cost and speed advantages, EST is ideal for a

first mode of contact in surveys, such that the researcher could begin with EST and use

progressively more expensive methods until enough responses are obtained. Or the

researcher could simply use EST among respondents having e-mail addresses and use

the postal technique to survey those without access. Likewise, one of the greatest

benefits of EST may be realized when it is used to send prior notifications and follow-

ups to increase responses to postal surveys and to EST itself.

EST can also be helpful in pretesting a survey instrument to increase the quality and

quantity of responses in a full-scale survey (Swoboda et al., 1997). The cost and speed

advantages of EST make it possible to conduct surveys aimed at establishing reliabil-

ity and validity of survey instruments. This initial process might also result in early

respondents commenting on the process of filling out the survey as well. Indeed, sev-

eral researchers have successfully used the pretest to get feedback for identifying the

optimal approach for conducting a mail survey (e.g., Hunt, Sparkman, & Wilcox,

1982).

Astudy by Clayton et al. (1996) demonstrates how EST can be used in combination

with other techniques, as well as for pilot surveying purposes. After developing the

survey instrument through two nominal group discussions, the researcher used the

EST to send the instrument for pilot survey purposes. They then sent a paper follow-up

survey to those who did not respond to the EST version. Once the survey instrument


was fully developed, the researcher used EST and then used a final paper-based survey

among those who did not respond to the electronic version. Through this mixed usage

of EST, the researchers were able to increase reliability of the survey instrument as

well as response rates while reducing the cost of the survey.

When a representative sample can be formed, it is apparent that EST can be used in

several types of organizational studies employing a self-administered data collection

technique. It can provide a good opportunity for those researchers who have a limited

research budget or who are interested in fast data gathering. Because e-mail obliterates

time and zone constraints, surveying with e-mail can prove very beneficial when the

sample is scattered or mobile or consists of members from such populations as execu-

tives who will not participate in personal or phone interviewing but may respond to an

e-mail questionnaire at their convenience. Indeed, EST has provided researchers with

the ability to reach rare, hidden, and geographically disperse populations (O’Lear,

1996; Sell, 1997). Moreover, because e-mail addresses are personal, sending the ques-

tionnaire to the right person can be more effective via EST than a mailed questionnaire

sent to a position wherein it is not always clear who is responding or usually results in

questionnaires being thrown away before reaching the person who has the required

information. That is, an e-mail survey intended for an individual is more likely to be

read and answered by that individual (Mehta & Sivadas, 1995). Likewise, compared to

other noncomputerized surveying techniques, EST is inexpensive, fast, and less prone

to many known sources of nonsampling errors such as data collection and data proc-

essing. In addition, compared to surveys over the Internet such as newsgroup surveys,

EST is the easiest to use and has better sampling control.

On the other hand, EST’s brimming potential is at present inhibited by its lack of

universal coverage, biased sampling frames, incompatibility of current e-mail sys-

tems, restricted binary file transfer, and technicalities involved in sending and receiv-

ing questionnaires. Of them, noncoverage error arguably presents the most significant

impediment to the increased use of EST. It makes EST unsuitable for conducting sur-

veys of many populations. There are no e-mail lists for most populations that can serve

as sampling frames, and constructing them can be very difficult, costly, and time-

consuming. Even when they exist, such frames are usually biased, primarily because

e-mail users by gender, age, race, income, education, and other major demographic

characteristics are very different from their populations, in a sense creating a unique

population. In particular, when the research project involves sample surveying of het-

erogeneous populations such as households, the researcher should be extremely cau-

tious in the decision to employ EST in isolation. We believe that until local sites on the

Internet develop and maintain local e-mail lists of general populations, e-mail lists will

usually suffer from being incomplete and outdated, much like traditional mailing lists

and telephone directories.

With respect to future research, we feel that the best research will likely come from

taking an interdisciplinary focus because EST is a multifaceted phenomenon. Within

this context, we feel that each component of the assessment framework should and

could be scrutinized for development of theoretical arguments. Meanwhile, empirical


research should investigate the comparative performance of EST, particularly vis-à-

vis postal surveying. So far, EST has mostly been used within organizations and edu-

cational institutions, so it will be fruitful and interesting to see findings from studies

conducted in different organizational and institutional settings. Some research must be

undertaken to explore different incentives to augment EST response rates because

with time, as the novelty of e-mail fades, reactions of computer users toward unsolic-

ited e-mail messages may become more negative. Because there is no way of knowing

whether approaches used to increase response rates are only initial stimulators, the

influences of these approaches on the quality of the data and the randomness of the

sample must be simultaneously investigated. Furthermore, we believe such issues

should be investigated through some experimentally designed studies. We urge

researchers to use a factorial experimental design to provide greater precision for esti-

mating overall variable effects, determine the interactions between the factors, and

allow the range of validity of the conclusions to be extended by the insertion of addi-

tional variables (Cox, 1992; Montgomery, 1991). In particular, fractionated-factorial

experiments allow a wide range of factors to be tested with small sample sizes (Box &

Hunter, 1961; Devor, Chang, & Sutherland, 1992). We finally urge researchers to

undertake research that focuses on EST as its primary goal rather than treating it as a

topic of secondary importance and making post hoc investigations and predictions

from project data that had another major agenda in mind. Unfortunately, many studies

that we reviewed on EST are of this type.

In sum, the rapid growth of global telecommunication networks, particularly the

Internet, has placed emphasis on EST as a surveying technique. EST is attractive

because it facilitates easy data management, location flexibility, and rapid transmis-

sion of the survey to all respondents across time and space. Yet, our review suggests

that it is too early to declare that EST has become a rival or a better technique than

major noncomputerized data collection techniques. Given current trends of rapidly

increasing e-mail availability, computer expertise, e-mail packages’compatibility, and

decreasing computer hardware and software cost, it is, however, conceivable that in

the near future, electronic surveying of many diverse populations will be possible.

Predicting this trend, several companies have recently introduced survey software

packages that work with e-mail systems to create, collect, and tabulate survey results.

In the long term, it seems that the most serious threats to EST will be legal ones and

potentially negative attitudes toward responding to a survey even in electronic format.

Aaker, D. A., Kumar, V., & Day, G. S. (1995). Marketing research. New York: John Wiley.

Albaum, G. (1987). Do source and anonymity affect mail survey results. Academy of Marketing

Science, 15, 74-81.

Anderson, S. E., & Gansender, B. M. (1995). Using electronic mail surveys and computer-

monitored data for studying computer-mediated communication systems. Social Science

Computer Review, 13, 33-46.

Ayidiya, S. A., & McClendon, J. M. (1990). Response effects in mail surveys. Public Opinion

Quarterly, 54, 229-247.

Bachmann, D., Elfrink, J., & Vazzana, G. (1996). Tracking the progress of e-mail vs. snail-mail.

Marketing Research, 8, 31-35.


Baker, R. P. (1992). New technology in survey research: Computer-assisted personal interview-

ing (CAPI). Social Science Computer Review, 10, 145-156.

Batinic, B. (1997). How to make an Internet based survey. Advances in Statistical Software, 6,

125-132.

Beebe, T. J., Mika, T., Harrison, P. A., Anderson, R. E., & Fulkerson, J. A. (1997). Computer-

ized school surveys. Social Science Computer Review, 15, 159-169.

Bloom, P. N., Milne, G. R., & Adler, R. (1995). Avoiding misuse of new information technolo-

gies: Legal and societal considerations. Journal of Marketing, 58, 98-110.

Box, G. E., & Hunter, J. S. (1961). The 2k-p factorial design part I. Technometrics, 3, 311-351.

Chadwick, B. A., Bahr, H. M., & Albrecht, S. L. (1984). Social science research methods.

Englewood Cliffs, NJ: Prentice Hall.

Childers, T. L., Pride, W. M., & Ferrell, O. C. (1980). Areassessment of the effects of appeals on

response to mail surveys. Journal of Marketing Research, 17, 365-370.

Church, A. H. (1993). Estimating the effect of incentives on mail survey response rates: A

meta-analysis. Public Opinion Quarterly, 57, 62-79.

Churchill, G. A. (1995). Marketing research methodological foundations. New York: Dryden.

Clayton, P., Applebee, A., & Pascoe, C. (1996). E-mail surveys: Old problems with a new deliv-

ery medium. LASIE, 27, 30-39.

Comley, P. (1996). The use of the Internet as a data collection method [Online]. Available:

http://www.sga.co.uk/esomar.html.

Corman, S. R. (1990). Computerized vs. pencil and paper collection of network data. Social

Networks, 12, 375-384.

Couper, M. P., & Burt, G. (1994). Interviewer attitudes toward computer-assisted personal inter-

viewing (CAPI). Social Science Computer Review, 12, 38-54.

Couper, M. P., & Rowe, B. (1996). Evaluation of a computer-assisted self-interview component

in a computer-assisted personal interview survey. Public Opinion Quarterly, 60, 89-105.

Cox, D. R. (1992). Planning of experiments. New York: John Wiley.

Dacko, S. G. (1995). Data collection should not be manual labor. Marketing News, 29, 31-35.

Daft, R. L., & Lengel, R. H. (1984). Information richness: Anew approach to managerial behav-

ior and organizational design. Research in Organizational Behavior, 6, 191-233.

Daft, R. L., Lengel, R. H., & Trevino, L. K. (1987). Message equivocality, media selection,

and managerial performance: Implications for information systems. MIS Quarterly, 11,

355-366.

Devor, R. E., Chang, T., & Sutherland, J. W. (1992). Statistical quality design and control: Con-

temporary concepts and methods. New York: Macmillan.

Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York: John

Wiley.

Dillman, D. A. (1991). The design and administration of mail surveys. Annual Review Sociol-

ogy, 17, 225-249.

Dillon, W. R., Madden, T. J., & Firtle, N. H. (1987). Marketing research in a marketing environ-

ment. Homewood, IL: Irwin.

Dyson, E. (1994). If nothing on e-mail is private, just say so. Computerworld, 27, 22, 33.

Emery, V. (1995). How to grow your business on the internet. New York: Coriolis.

Erdos, P. L. (1983). Professional mail surveys. Malabar, FL: Krieger.

Fox, R. J., Crask, M. R., & Kim, J. (1988). Mail survey response rate: A meta analysis of se-

lected techniques for inducing response. Public Opinion Quarterly, 52, 229-247.

Gaiser, T. J. (1997). Conducting on-line focus groups. Social Science Computer Review, 15,

135-144.

Garton, L., & Wellman, B. (1995). Social impacts of electronic e-mail in organizations: A re-

view of the research literature. In B. R. Burleson (Ed.), Communication yearbook (Vol. 18,

pp. 434-452). Thousand Oaks, CA: Sage.


Gates, R. H., & Jarboe, G. R. (1987). Changing trends in data acquisition for marketing research.

Journal of Data Collection, 27, 25-29.

Gilbert, G. H., Longmate, J., & Branch, L. G. (1992). Factors influencing mailed health surveys.

Public Health Reports, 107, 576-584.

Goree, C. T., & Marszalek, J. F., III. (1995). Electronic surveys: Ethical issues for researchers.

College Student Affairs Journal, 15, 75-79.

Gwyne, S. C., & Dickerson, J. F. (1997, April 21). Lost in the e-mail. Time, pp. 88-90.

Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis

with readings. Englewood Cliffs, NJ: Prentice Hall.

Havice, M. J. (1990). Measuring non-responses and refusals to an electronic survey. Journalism

Quarterly, 67, 521-530.

Heberlin, T. A., & Baumgartner, R. (1978). Factors affecting response rate to mailed question-

naires: A quantitative analysis of the published literature. American Sociological Review,

3, 447-462.

Helliwell, J. (1986). Global standards promise fluency in electronic mail. PC Week, 3, 73-77.

Higgins, C. A., Dimnik, T. P., & Greenwood, H. P. (1987). The DISKQ survey method. Journal

of the Market Research Society, 29, 437-445.

Hunt, S. D., Sparkman, R. D., & Wilcox, J. (1982). The pretest in survey research: Issues and

preliminary findings. Journal of Marketing Research, 19, 269-273.

Jobber, D. (1986). Improving response rates in industrial mail surveys. Industrial Marketing

Management, 15, 183-195.

Jobber, D., & Sanderson, S. M. (1985). Evaluating the effectiveness of two variables on indus-

trial mail survey response rates. Industrial Marketing Management, 14, 119-121.

Johnston, J., & Walton, C. (1995). Reducing response effects for sensitive questions: A

computer-assisted self interview with audio. Social Science Computer Review, 13,

304-319.

Kaldenberg, D. O. (1994). Mail survey response patterns in a population of elderly: Does re-

sponse deteriorate with age? Public Opinion Quarterly, 58, 68-76.

Kalton, G. (1983). Compensating for missing survey data. Ann Arbor: University of Michigan

Press.

Kambil, A. (1995). Electronic commerce: Implications of the Internet for business practice and

strategy. Business Economics, 30, 27-32.

Kantor, J. (1991). The effects of computer administration and identification on the Job Descrip-

tive Index (JDI). Journal of Business & Psychology, 5, 309-323.

Karweit, N., & Meyers, E. D. (1983). Computers in survey research. In P. H. Rossi, J. D. Wright, &

A. B. Anderson (Eds.), Handbook of survey research (pp. 379-413). New York: Academic

Press.

Kerr, E. B., & Hiltz, S. R. (1982). Computer-mediated communication systems. New York:

Academy Press.

Kiesler, S. (1989). E-mail brings out the best and worst in group interaction. Infoworld, 4-9.

Kiesler, S., & Sproull, L. S. (1986). Response effects in the electronic survey. Public Opinion

Quarterly, 50, 402-413.

Kiesler, S., Zubrow, D., & Moses, A. M. (1985). Affect in computer-mediated communication:

An experiment in synchronous terminal-to-terminal discussion. Human-Computer Inter-

action, 1, 77-104.

Kittleson, M. J. (1995). An assessment of the response rate via the postal service and e-mail.

Health Values, 18, 27-29.

Kittleson, M. J. (1997). Determining effective follow-up of e-mail surveys. American Journal

of Health Behavior, 21, 193-196.

Kochmer, J., & Northwest. (1993). Northwestnet’s guide to our world Internet (4th ed.). Wash-

ington, DC: Northwest Academic Computing Consortium.


Komsky, S. H. (1991). A profile of users of electronic mail in a university. Management Com-

munication Quarterly, 4, 310-340.

Lavrakas, P. C. (1996). To err is human. Marketing Research, 8, 30-36.

Leeuw, E. D., Hox, J. J., & Snijkers, G. (1995). The effect of computer-assisted interviewing on

data quality: A review. Journal of the Market Research Society, 37, 325-335.

Liefeld, J. P. (1988). Response effects in computer-administered questioning. Journal of Mar-

keting Research, 25, 405-409.

Linsky, A. S. (1975). Stimulating responses to mailed questionnaires: Areview. Public Opinion

Quarterly, 3, 82-101.

Lockhart, D. C., & Russo, J. R. (1996). Mail and telephone surveys in marketing research: A

perspective from the field. In R. P. Bagozzi (Ed.), Principles of marketing research (pp. 117-

161). Cambridge, UK: Blackwell.

Malhotra, N. K. (1993). Marketing research: An applied orientation. Englewood Cliffs, NJ:

Prentice Hall.

Markus, M. L. (1994). Electronic mail as the medium of managerial choice. Organization Sci-

ence, 5, 502-527.

Mehta, R., & Sivadas, E. (1995). Comparing response rates and response content in mail versus

electronic mail surveys. Journal of the Market Research Society, 37, 429-439.

Montgomery, D. C. (1991). Design and analysis of experiments. New York: John Wiley.

Mortensen, E. (1985). Electronic mail: An ever changing technology. The Office, 102, 101-103.

Neal, W. D. (1989). Computering and software in marketing research: The US perspective.

Journal of the Market Research Society, 31, 567-576.

O’Lear, S.R.M. (1996). Using electronic mail (e-mail) surveys for geographic research: Les-

sons from a survey of Russian environmentalists. Professional Geographer, 48, 209-217.

Oppermann, M. (1995). E-mail surveys: Potential and pitfalls. Marketing Research: A Maga-

zine of Management & Applications, 7, 29-33.

Parker, L. (1992). Collecting data the e-mail way. Training & Development, 46, 52-54.

Parker, T. (1995). How internet e-mail works. In M. A. Pike (Ed.), Using the Internet (pp. 283-322).

Indianapolis, IN: Que.

Paxson, M. C. (1995). Increasing survey response rates: Practical instructions from the total-

design method. Cornell Hotel & Restaurant Administration Quarterly, 36, 66-74.

Perl, Y., & Whitley, E. (1995, December). The environmental contribution of personal comput-

ers: A state of the art report. Computers and Society, pp. 6-8.

Rice, R. E. (1990). Computer-mediated communication system network data: Theoretical

concerns and empirical examples. International Journal of Man-Machine Studies, 32,

627-647.

Rossi, P. H., Wright, J. D., & Anderson, A. B. (1983). Sample surveys: History, current practice,

and future prospects. In P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook of

survey research (pp. 1-20). New York: Academic Press.

Roth, P. L., & BeVier, C. A. (1998). Response rates in HRM/OB survey research: Norms and

correlates, 1990-1994. Journal of Management, 24, 97-118.

Salant, P., & Dillman, D. A. (1994). How to conduct your own survey. New York: John Wiley.

Saltzman, A. (1995). Improving response rates in disk-by-mail surveys. Marketing Research: A

Magazine of Management & Applications, 5, 32-39.

Schaefer, D. R., & Dillman, D. A. (1998). Development of standard e-mail methodology: Re-

sults of an experiment. Public Opinion Quarterly, 62, 378-386.

Schuldt, B. H., & Totten, J. W. (1994). Electronic mail vs. mail survey response rate. Marketing

Research: A Magazine of Management & Applications, 6, 36-44.

Sell, R. L. (1997). Research and the Internet: An e-mail survey of sexual orientation. American

Journal of Public Health, 2, 297.

Shannon, J. H., & Rosenthal, D. A. (1993). Electronic mail and privacy: Can the conflicts be re-

solved? Business Forum, 18, 31-42.


Sproull, L. S. (1986). Using electronic mail for data collection in organizational research. Acad-

emy of Management Journal, 29, 159-169.

Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel

Psychology, 51, 709-725.

Subramanian, A. K., McAfee, A. T., & Getzinger, J. P. (1997). Use of the World Wide Web for

multisite data collection. Academic Emergency Medicine, 4, 811-817.

Sudman, S. (1996). Sampling. In R. P. Bagozzi (Ed.), Principles of marketing research (pp. 73-

115). Cambridge, UK: Blackwell.

Swoboda, W. J., Muhlberger, N., Weitkunat, R., & Scheneeweib, S. (1997). Internet surveys by

direct mailing: An innovative way of collecting data. Social Science Computer Review, 15,

242-255.

Synodinos, N. E., & Brennan, J. M. (1988). Computer active interviewing in survey research.

Psychology & Marketing, 5, 117-137.

Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collec-

tion mode, question format, and question context. Public Opinion Quarterly, 60, 275-304.

Tse, A.C.B. (1998). Comparing the response rate, response speed and response quality of tow

methods of sending questionnaires: E-mail vs. mail. Journal of the Market Research Soci-

ety, 40, 353-361.

Tull, D. S., & Hawkins, D. I. (1993). Marketing research: Measurement & method. New York:

Macmillan.

Vazzana, G. S., & Bachmann, D. (1994). Fax attracts. Marketing Research: AMagazine of Man-

agement & Applications, 6, 19-25.

Veiga, J. F. (1984). Getting the mail questionnaire returned: Some practical research considera-

tions. Journal of Applied Psychology, 59, 217-238.

Walsh, J. P., Kiesler, S., Sproull, L. S., & Hesse, B. W. (1992). Self-selected and randomly se-

lected respondents in a computer network survey. Public Opinion Quarterly, 56, 241-244.

Watson, S. C. (1998). Aprimer in survey research. Journal of Continuing Higher Education, 46,

31-40.

Weisband, S., & Kiesler, S. (1996). Self disclosure on computer forms: Meta analysis and impli-

cations. Human Factors in Computing Systems Conference Proceedings, 3-10.

Yammarino, F. J., Skinner, S. J., & Childers, T. L. (1991). Understanding mail survey response

behavior: A meta-analysis. Public Opinion Quarterly, 55, 613-639.

Yu, J., & Cooper, H. (1983). Aquantitative review of research design effects on response rates to

questionnaires. Journal of Marketing Research, 20, 36-44.

Zikmund, W. G. (1994). Business research methods. New York: Harcourt Brace.

Zeki Simsek is a graduate student in the School of Business Administration, University of Connecticut. His

research interests include new data collection techniques, intraorganizational networks, and interorganiza-

tional relations.

John F. “Jack” Veiga (DBA Kent State) is the Airbus Industrie International Scholar and head of the Man-

agement Department, School of Business Administration, University of Connecticut. His current research

interests include technology acceptance, cross-cultural behavior, and top management teams.


Documents

ZEKI SIMSEK JOHN F. VEIGA - University of Crete