Research About Survey via E-mail

8/14/2019 Research About Survey via E-mail

1/25

RESPONSE VARIATION IN E-MAIL SURVEYS: AN EXPLORATION

by Kim Bartel Sheehan

A i tant Pr!"e !r# Uni$er ity !" Ore%!n

an& Sally '( M)Millan

A i tant Pr!"e !r# Uni$er ity !" Tenne ee

*ire)t )!rre +!n&en)e t! the "ir t a,th!r at:

Kim Sheehan

./0 Uni$er ity !" Ore%!nE,%ene# OR 1/234

5 heehan6!re%!n(,!re%!n(e&,Ph!ne: 02 -427-.3889a : 02 -427-427.

The a,th!r ;i h t! than5 Mariea er "!r their %,i&an)e

in )!n)e+t,ali>in% an& )!n&,)tin% the re ear)h +r!?e)t re+!rte& in thi t,&y(


2/25

Re +!n e Variati!n in E-Mail S,r$ey

RESPONSE VARIATION IN E-MAIL SURVEYS: AN EXPLORATION

Ab tra)t

A e-mail an& !ther relate& te)hn!l!%ie ha$e &i"", e& ra+i&ly int! a lar%e an&

heter!%ene!, +!+,lati!n# re ear)her ha$e be%,n t! e +l!re the +! ibility !" , in% e-mail a a

t!!l "!r ,r$ey re ear)h(

,r$ey meth!& nee& t! &e$el!+ the 5in& !" ri)hne that i "!,n& in the literat,re !n tra&iti!nal

+! tal mail ,r$ey meth!& ( Thi arti)le ta5e a "ir t te+ in &e$el!+in% that literat,re ;ith an in-

&e+th )!m+ari !n !" three t,&ie that , e& e-mail ,r$ey "!r &ata )!lle)ti!n( *etail are

+r!$i&e& !n meth!& "!r am+lin% an& ,r$ey te)hni@,e ( e& relati!n hi+ bet;een

i ,e alien)e an& re +!n e rate# an& bet;een +re- ,r$ey n!ti"i)ati!n an& re +!n e time# ;ere

%enerally ,++!rte&(

.


3/25


Re ear)her in$e ti%ati!n !" )!m+,ter-me&iate& )!mm,ni)ati!n a a t!!l "!r )!n&,)tin%

re ear)h an& )!lle)tin% )!n ,mer &ata ha been !n the in)rea e a Internet , a%e am!n% +e!+le

ar!,n& the ;!rl& )!ntin,e t! %r!;( T!&ay# a many a 33 milli!n +e!+le ;!rl&;i&e ha$e

a))e t! e-mail *O=# 118C# 83 +er)ent !" all , er l!% !n t! the Internet !n a &aily ba i # an&

the &em!%ra+hi) +r!"ile !" Internet , er in the Unite& State i be%innin% t! mirr!r that !" the

%eneral +!+,lati!n Keh!e# Pit5!; an& M!rt!n# 11/C( *ata )!lle)te& $ia h!me +a%e ,r$ey !n

the D!rl& Di&e Deb ,)h a the e!r%ia Te)h t,&ie C are the m! t +,bli)i>e& e""!rt "!r

)!lle)tin% in"!rmati!n $ia the Internet(

analy>e the , e !" ele)tr!ni) mail e-mailC t! &i eminate ,r$ey an& )!lle)t &ata( Re ear)h int!

the $iability !" e-mail a a ,r$ey meth!& ha "!), e& +rimarily !n )!m+arin% re +!n e rate an&

re +!n e +ee& !" e-mail ,r$ey t! th! e !" +! tal mail( O$erall# the e t,&ie ,%%e t that e-

mail ha %reat +!tential "!r ,r$ey re ear)her (

Re ear)her ha$e re+!rte& a ;i&e $ariati!n in re +!n e rate an& +ee& !" re +!n e "!r e-

mail ,r$ey ee Table C( Thi i n!t ,r+ri in% be)a, e !" the $ariety !" am+le +!+,lati!n an&

re ear)h t!+i) re+!rte& in th! e t,&ie ( A re ear)her +lannin% an e-mail ,r$ey t!&ay ha

minimal in"!rmati!n !n ;hi)h t! ba e e timati!n !" re +!n e rate an& there"!re ;ill ha$e

&i""i),lty in &eterminin% am+le i>e( The limite& +,bli he& re ear)h !n e-mail meth!&!l!%y al !

+r!$i&e little in"!rmati!n t! a i t re ear)her ;ith !ther ba i) re ear)h &e i%n i ,e ,)h a

@,e ti!nnaire &e$el!+ment an& re +!n&ent )!nta)t ( Re ear)her ha$e a ;ealth !" in"!rmati!n !n

re +!n e e""e)t "!r +! tal mail ,r$ey b,t the literat,re a&&re in% ,)h e""e)t "!r e-mail

,r$ey i minimal(

The +,r+! e !" thi t,&y i t! +r!$i&e re ear)her ;ith in"!rmati!n that )an a i t in the

&e i%n an& im+lementati!n !" e-mail ,r$ey re ear)h( The literat,re !n re +!n e e""e)t in +! tal

4


4/25


mail ,r$ey +r!$i&e a "rame;!r5 "!r &i ), i!n !" 5ey &e i%n i ,e ( Sam+lin% an& ,r$ey

te)hni@,e "!r three t,&ie that , e& e-mail ,r$ey are &e )ribe& in &etail( 9inally# ;e e amine

the im+a)t !" t!+i) alien)e an& +re-n!ti"i)ati!n# t;! 5ey +re&i)t!r !" re +!n e in a +! tal mail

,r$ey # !n re +!n e rate !" e-mail ,r$ey (

Re$ie; !" the Literat,re

E-Mail S,r$ey

E-mail ha been )hara)teri>e& a a G+r!mi in% mean "!r )!n&,)tin% ",t,re ,r$ey H

S)hae"er an& *illman# 118C# an& n,mer!, re ear)her ha$e re)!%ni>e& the bene"it that e-mail

+r!$i&e !$er +! tal mail( The e bene"it in)l,&e )! t a$in% "r!m eliminati!n !r re&,)ti!n !"

+a+er )! t an& mailin% )! t Par5er# 11.C an& the ra+i& +ee& !" re +!n e Ba)hmann# El"rin5

an& Va>>ana# 117F Mehta an& Si$a&a # 110C( In "a)t# a )!n i tent "in&in% !" the t,&ie that

)!m+are re +!n e +ee& !" ,r$ey &eli$ere& $ia e-mail an& +! tal mail i that e-mail re +!n e

are ret,rne& m,)h m!re @,i)5ly than +! tal mail re +!n e Ba)hmann# El"rin5 an& Va>>ana#

117F Kie ler an& S+r!,ll# 187F S)hae"er an& *illman# 118F Deible an& Dalla)e# 118C( In

the e t,&ie # e-mail re +!n e +ee& ran%e& "r!m "i$e t! ten &ay # )!m+are& t! the re +!n e

+ee& !" +! tal mail ,r$ey # ;hi)h ran%e& "r!m ten t! "i"teen &ay ee Table C(

Re +!n e rate t! e-mail ,r$ey # h!;e$er# &! n!t )!n i tently h!; bene"it !$er +! tal

mail# an& in !me )a e "all bel!; ;hat may be een a a))e+table le$el !" re +!n e( Kie ler an&

S+r!,ll 187C an& Par5er 11.C re+!rte& e-mail re +!n e rate !" !$er 70 +er)ent# ;ith b!th

t,&ie h!;in% e-mail re +!n e rate i%ni"i)antly hi%her than the )!m+arable +! tal mail

meth!&( S)hae""er an& *illman 118C an& Mehta an& Si$a&a 117C "!,n& n! i%ni"i)ant

&i""eren)e in re +!n e rate bet;een the t;! m!&e ( Se$eral !ther t,&ie e(%( S)h,l&t an&

T!tten# 112F T e et al# 110F Deible an& Dalla)e# 118C "!,n& that e-mail re +!n e rate ;ere

2


5/25


l!;er than th! e !" +! tal mail( Re +!n e rate "!r e-mail ,r$ey ee Table C $ary "r!m a l!;

!" 7 +er)ent T e et al# 110C t! a hi%h !" /0 +er)ent Kie ler an& S+r!,ll# 187C(

The e &i""eren)e in re +!n e rate are n!t ,r+ri in% %i$en ;hat i 5n!;n ab!,t re +!n e

e""e)t in +! tal mail ,r$ey ( The t,&ie h!;n in Table ha$e h!m!%ene!, am+le # mall

am+le i>e # an& &i$er e ,r$ey t!+i) ( The ty+e !" am+le +!+,lati!n are either em+l!yee !" a

in%le )!m+any , e& in t;! t,&ie C !r Uni$er ity +r!"e !r an& *ean , e& in "i$e t,&ie C#

;ith !nly !ne t,&y )!n i tin% !" a am+le !" Internet , er Mehta an& Si$a&a # 110C( S,r$ey

t!+i) ran%e& "r!m )!r+!rate an& Internet )!mm,ni)ati!n t! b, ine ethi) an& T M( i$en the

la)5 !" )!n i ten)y in n,mer!, $ariable in the e t,&ie # the ran%e !" re +!n e rate an& +ee&

i ,n&er tan&able(

Dhat i mi in% "r!m the ),rrent b!&y !" re ear)h i a )!m+ari !n !" e-mail ,r$ey

re +!n e bey!n& the im+le )!m+ari !n t! re +!n e rate !" +! tal mail ,r$ey ( The b!&y !"

5n!;le&%e ab!,t +! tal mail ,r$ey meth!&!l!%y ,%%e t a n,mber !" i ,e that m, t be

)!n i&ere& &,rin% the &e i%n an& im+lementati!n !" a +! tal ,r$ey an& that ha$e the +!tential t!

e""e)t re +!n e rate an& +ee&( The e e""e)t may al ! be rele$ant "!r e-mail ,r$ey (

P! tal Mail S,r$ey

A re$ie; !" the rele$ant literat,re re%ar&in% +! tal mail meth!&!l!%y ,%%e t that

n,mer!, &e i%n an& im+lementati!n i ,e may e""e)t b!th re +!n e rate an& +ee& in thi m!&e(

The literat,re i ri)h in meta-analy e that +r!$i&e in&i)ati!n !" ,)h e""e)t # an& many !" the e

i ,e ;ill al ! be rele$ant "!r e-mail t,&ie ( The literat,re ha re+!rte& the "!ll!;in% e""e)t in

+! tal mail ,r$ey :

Per !nali>ati!n !" )!$er letter( Per !nali>in% letter a&&re e& t! +e)i"i) in&i$i&,al ha

been h!;n t! in)rea e re +!n e rate in !me +! tal mail ,r$ey *illman# 1/8F 11 C# ;hile

!ther *,n)an# 1/1C "!,n& n! e""e)t !n re +!n e rate &,e t! )!$er letter +er !nali>ati!n( In e-

0


6/25


7/25


Yammarin!# S5inner an& =hil&er 11 C &i& n!t "in& ,++!rt "!r the $al,e !" a Uni$er ity

+!n !r hi+ t! e""e)t re +!n e rate( S+!n !r hi+ !" e-mail ,r$ey )ann!t be a e +li)it a ;ith

+! tal mail ,r$ey i(e( the , e !" a +!n !rin% !r%ani>ati!n letterhea& i n!t a$ailableC# b,t

+!n !r hi+ )an be ma&e im+li)itly thr!,%h tatement in the ,r$ey in tr,ment an& thr!,%h the

en&er e-mail a&&re e i(e( an G(e&,H ,""i !n an a&&re ;!,l& in&i)ate a !)iati!n ;ith an

e&,)ati!nal in tit,ti!nC(

,e ti!nnaire &e i%n( *e i%n i ,e # ,)h a the len%th !" the @,e ti!nnaire# )an e""e)t

re +!n e( The l!n%er the @,e ti!nnaire# the le li5ely +e!+le are t! re +!n& !" )!n"i&entiality by in"!rmin% them that their e-mail a&&re e ;ill n!t be re)!r&e& ;ith their

,r$ey re +!n e an& that the ,r$ey &ata ;ill be )!n i&ere& !nly in the a%%re%ate(

I ,e Salien)e( In +! tal mail ,r$ey # the alien)e !" an i ,e t! the am+le& +!+,lati!n

ha been "!,n& t! ha$e a tr!n% +! iti$e )!rrelati!n ;ith re +!n e rate( Salien)e ;a &e"ine& a

t!+i) that &ealt ;ith an im+!rtant i ,e that ;a al ! ),rrent !r timely Martin# 110C(


8/25


an& Ba,m%artner 1/8C "!,n& that i ,e alien)e ha& a tr!n%er im+a)t !n re +!n e rate than

&i& any !ther i ,e !r re ear)h &e i%n &e)i i!n in)l,&in% a&$an)e n!ti)e# "!ll!;-,+ )!nta)t # !r

m!netary in)enti$e ( R!ber !n an& San& t!rm 113C an& Martin 110C al ! "!,n& that alien)e

;a a 5ey +re&i)t!r !" re +!n e rate "!r +! tal mail ,r$ey ( Un&er tan&in% the +!+,lati!n t! be

am+le& i an im+!rtant "ir t te+ in &eterminin% i ,e alien)e( Re ear)her ;h! , e e-mail

,r$ey may be able t! be%in t! +re&i)t re +!n e rate !n the ba i !" h!; alient an i ,e i t! the

in&i$i&,al ;h! ;ill be !li)ite& t! +arti)i+ate in the e-mail ,r$ey(

Re +!n&ent =!nta)t ( 9! # =ra 5 an& Kim 188C "!,n& that +re-n!ti"i)ati!n by letter le&

t! in)rea e in re +!n e rate "!r +! tal mail ,r$ey (

1/8C "!,n& little !r n! e""e)t a !)iate& ;ith +re-n!ti"i)ati!n( Se$eral t,&ie !" +! tal mail

,r$ey Kan,5 an& Beren !n# 1/0F M,r+hy# *aley an& *alenber%# 11 F Tayl!r an& Lynn#

118C "!,n& re +!n e +ee& ;a "a ter "!r +re-n!ti"ie& re +!n&ent than "!r th! e ;h! ;ere n!t

+re-n!ti"ie&( Yammarin!# S5inner an& =hil&er 11 C ,%%e te& that "!ll!;-,+ mailin% an&

re+eate& )!nta)te& eeme& t! ha$e a %reater e""e)t !n re +!n e rate am!n% th! e ;h! re)ei$e

the ,r$ey be)a, e !" an in tit,ti!nal a""iliati!n than am!n% th! e ;h! re)ei$e a %eneral )!n ,mer

,r$ey( Little )!n en , ;a "!,n& !n the $al,e !" m,lti+le +re- an& +! t- ,r$ey )!nta)t in

+! tal mail-ba e& ,r$ey ( Re ear)her , in% +! tal mail "!r &eli$ery !" me a%e m, t ;ei%h the

+!tential bene"it !n re +!n e rate a%ain t the )! t !" m,lti+le mailin% (

Be)a, e +ee& !" re +!n e ha been een a a 5ey bene"it t! e-mail ,r$ey # enhan)in%

re +!n e +ee& i im+!rtant "!r re ear)her ;h! ;i h t! ma imi>e the +!tential !" the m!&e( An&

be)a, e m! t re ear)her )an en& m,lti+le e-mail me a%e "!r little !r n! )! t# the im+a)t !"

m,lti+le )!nta)t !n re +!n e be)!me a hi%hly rele$ant ,b?e)t "!r e-mail ,r$ey (

8


9/25



10/25


the alien)e "!r thi %r!,+ ;a the "a)t that the in&i$i&,al )!lle)tin% the in"!rmati!n ;a a t,&ent

at the ,ni$er ity ;here the re +!n&ent ;ere a""iliate&( The t!+i) !" St,&y 4 ;a n!t alient t! the

,b?e)t +!+,lati!n( It ;a +re ente& a a G&!)t!ral t,&ent ,r$eyH !" Internet , a%e habit (

Se)!n the t,&ie &i""er in term !" +re-n!ti"i)ati!n( St,&y &i& n!t +re-n!ti"y

re +!n&ent # St,&ie . an& 4 &i& en& a +re-n!ti"i)ati!n me a%e( Thi +re-n!ti"i)ati!n e-mail

me a%e e +laine& the +,r+! e !" the re ear)h an& n!ti"ie& ,b?e)t that they ;!,l& re)ei$e a

,r$ey ;ithin a &e i%nate& time +eri!&( S,b?e)t ;ere t!l& that they )!,l& &e)line t! +arti)i+ate

by re+lyin% t! thi "ir t me a%e an& a 5in% that the ,r$ey n!t be ent( Thi te)hni@,e i imilar t!

&ire)t mar5etin% +ra)ti)e , e& by !r%ani>ati!n ,)h a b!!5 an& re)!r& )l,b that &e"a,lt t!

en&in% an item ,nle the )!n ,mer &e)line ( Le than t;! +er)ent !" the ,ni$er ity "a),lty#

ta"" an& t,&ent !" St,&y . &e)line& t! +arti)i+ate( Sli%htly m!re than +er)ent !" the

in&i$i&,al ;ith +er !nal e-mail a))!,nt in St,&y 4 &e)line& t! +arti)i+ate in thi ,r$ey(

Be)a, e the +r!)e&,re "!r )!n&,)tin% e-mail ,r$ey are relati$ely ne;# a&&iti!nal

meth!&!l!%i)al in"!rmati!n i +r!$i&e& t! %,i&e re ear)her ;h! ;i h t! , e thi te)hni@,e( In

+arti),lar# ;e +r!$i&e in"!rmati!n !n am+lin% an& ,r$ey te)hni@,e , e& "!r the e three t,&ie (

Sam+lin%

St,&y Deb-Site *e$el!+er C( The Yah!! &ire)t!ry !" health-relate& Deb ite ;a

, e& a the ,ni$er e "!r St,&y ( Yah!! &!e n!t e@,entially n,mber ite ;ithin it )ate%!rie (

There"!re# a trate%y ;a nee&e& "!r i&enti"yin% the n,mber !" ite in the ,ni$er e# ran&!mly

ele)tin% ite # an& &eterminin% h!; t! a++ly ran&!m n,mber t! +e)i"i) ite ( The i>e !" the

,ni$er e ;a &etermine& by a&&in% t!tal that Yah!! re+!rte& "!r ea)h health-relate& )ate%!ry !n

the G!+enin% +a%eH !r in&e C !" the Yah!! )ate%!ry li tin% ( At the time !" the t,&y# the

n,mber "!ll!;in% ea)h !" the e )ate%!rie ;ere a&&e& t! !btain the ,ni$er e i>e !" 2#/12(

3


11/25


A li t !" #303 ,ni@,e ran&!m n,mber ran%in% "r!m t! 2#/12 ;a &ra;n( The e

n,mber ;ere a++lie& by tartin% at the t!+ !" the "ir t !n-line +a%e !" Yah!! li tin% an& ;!r5in%

thr!,%h t! the b!tt!m !" that +a%e( 9!r e am+le# the "ir t item !n the "ir t +a%e !" the health

)ate%!ry ;a a t!+i) li tin% "!r Alternati$e Me&i)ine( Dhen that li tin% ;a ele)te an!ther

)reen a++eare& ;hi)h in)l,&e& a&&iti!nal ,b-)ate%!rie e(%(# in tit,te # ma%a>ine # et)(C a ;ell

a &ire)t lin5 t! ite ( S,b-)ate%!rie ;ere "!ll!;e& ,ntil they re ,lte& in a ite li tin%(

9!,r "a)t!r le& t! re&,)ti!n !" the am+le i>e( 9ir t# !me ite ha& t! be eliminate&

be)a, e they &,+li)ate& either the URL ,ni"!rm re !,r)e l!)at!r J the a&&re !" the iteC !r the

e-mail a&&re !" an!ther ite( In ea)h )a e# the "ir t ite ;ith the &,+li)ate URL an& !r e-mail

a&&re ;a 5e+t an& the e)!n& ;a &i )ar&e&( Se)!n !me URL li te& by Yah!! ;ere n!t

",n)ti!nin%( Thir !me ite &i& n!t ha$e an e-mail a&&re ( 9inally# in !me )a e an e-mail

a&&re ;a "!,n& b,t e-mail ;a ,n&eli$erableF 8(7 +er)ent !" the ele)te& ite )!,l& n!t be

)!nta)te& be)a, e !" mi in% !r n!n-",n)ti!nin% e-mail a&&re e ( A"ter a&?, tin% "!r the ab!$e

"a)t!r # the t!tal $ali& am+le i>e "!r St,&y ;a 842(

St,&y . Uni$er ity 9a),lty# Sta"" an& St,&ent C( The ,ni$er ity &ire)t!rie !" "a),lty# ta""

an& t,&ent +r!$i&e& the am+le "!r St,&y .( The e &ire)t!rie +r!$i&e name # a&&re ee #

+h!ne n,mber # an ;here a$ailable# e-mail an& Deb +a%e a&&re e "!r in&i$i&,al li te&( A

t!tal !" 083 name ;ere ele)te&F t;! thir& 487C ;ere "r!m the "a),lty ta"" &ire)t!ry an& !ne-

thir& 12C ;ere "r!m the t,&ent &ire)t!ry( A ran&!m n,mber ;a , e& t! ele)t the "ir t +a%e

"r!m ;hi)h t! &ra; name ( Be%innin% !n that +a%e# e$ery ei%hth name ;a ele)te& ,ntil the

&e ire& n,mber !" name ;a rea)he&( I" the ei%hth name &i& n!t ha$e an e-mail a&&re it ;a

5i++e& an& re+la)e& ;ith the ne t entry that in)l,&e& an e-mail a&&re (

St,&y 4 In&i$i&,al ;ith Per !nal E-mail A))!,nt C( A t;!- ta%e am+lin% te)hni@,e ;a

, e& "!r St,&y 4( 9ir t# the re ear)her ran&!mly ele)te& Internet Ser$i)e Pr!$i&er "r!m an !n-


12/25


line li t( *,rin% thi ele)ti!n# the re ear)her )he)5e& t! en ,re that the ISP +r!$i&e& er$i)e t!

in&i$i&,al rather than b, ine e ( A t!tal !" 00 ISP ;ere ele)te& in thi ta%e(

In the e)!n& ta%e# the ISP &!main name e(%( earthlin5()!mC ;a entere& in the 9!,r

ear)h en%ine( Thi ear)h en%ine i &e i%ne& t! "in& in&i$i&,al e-mail a&&re e ( The ear)h

en%ine )!,l& ear)h , in% any )!mbinati!n !" "ir t name# la t name# &!main name# tate# an&

)!,ntry( Sin)e the time that thi t,&y ;a ,n&erta5en# the 9!,r ear)h en%ine ;a !l& t!

Yah!!( The ear)h +arameter all!;e& by the er$i)e ha$e been )han%e&C(

9!r St,&y 4# the !nly ear)h +arameter ;ere that in&i$i&,al m, t be in the Unite& State

an& that they m, t ha$e ha& an a))!,nt ;ith a +e)i"i) ISP( Dith the e +arameter # 9!,r

+r!&,)e& a li t !" +er !n ;h! ha& an e-mail a&&re ;ith the +e)i"ie& ISP( I" .33 !r "e;er

in&i$i&,al , e& that ISP "!r e-mail# all e-mail a&&re e ;ere +r!$i&e&( I" m!re than .33

in&i$i&,al ha& e-mail a))!,nt ;ith the ISP# a ran&!m am+le !" .33 e-mail a&&re e ;a

%enerate&( In thi )a e# the t!tal n,mber !" in&i$i&,al ;ith e-mail a&&re e ;a al ! re+!rte&(

E-mail li t +r!$i&e& by the 9!,r ear)h en%ine ;ere am+le& a nee&e& t! en ,re that

"!r ea)h ISP the ,r$ey re)i+ient ;!,l& be &ra;n at ran&!m( Three ty+e !" am+lin% ;ere , e&(

9ir t# i" the 9!,r ear)h en%ine re+!rte& that m!re than #333 e-mail a&&re e ;ere +r!$i&e&

by the ISP# then the .33 ran&!m e-mail a&&re e ret,rne& by 9!,r ;ere , e&( Se)!n i" the

9!,r ear)h en%ine ret,rne& .33 ran&!m e-mail a&&re e b,t re+!rte& that #333 !r "e;er

in&i$i&,al ha& an e-mail a))!,nt ;ith the ISP# ;e "elt it ne)e ary t! am+le( In thi )a e# !ne-

"!,rth !" the .33 e-mail a&&re e ;a ele)te& at ran&!m( 9inally# i" "e;er than .33 e-mail

a&&re e ;ere ret,rne ;e am+le& "r!m the li t +r!$i&e& by 9!,r ( One-"!,rth !" the e

a&&re e ;ere ele)te& at ran&!m( U in% the e am+lin% te)hni@,e # 0#333 e-mail a&&re e ;ere

ele)te& t! re)ei$e the ,r$ey( Pre-te t in&i)ate& that a ,b tantial +er)enta%e !" e-mail a&&re e

;!,l& n! l!n%er be a)ti$e( In thi )a e# #./7 !" the e-mail a&&re e ;ere n!t &eli$erable(

.


13/25


S,r$ey Te)hni@,e

A +re$i!, ly menti!ne many tan&ar& ,r$ey te)hni@,e ;ere , e& in all three t,&ie (

All ,b?e)t ;ere %i$en the !+ti!n n!t t! re +!n&( =!n"i&entiality !" re +!n&ent ;a a ,re&(

The intr!&,)ti!n t! the ,r$ey in)l,&e& an e timate !" h!; m,)h time ;!,l& be re@,ire& "!r

)!m+letin% the ,r$ey(

3 time a m,)h a e-mail ,r$ey ( Three a&&iti!nal te)hni@,e are &i""erent in the e-mail ,r$ey

"!rmat: en&in% e-mail# remin&er # an& re +!n&in%(

Sen&in% E-mail( 9!r St,&y # in&i$i&,al me a%e ;ere ent t! ea)h ,b?e)t( E-mail

a&&re e ;ere )!+ie& t! the e-mail me a%e "r!m a +re$i!, ly )reate& &ataba e( Dhile thi ;a

time )!n ,min%# it enable& the re ear)her t! 5ee+ re)!r& !" ;hen me a%e ;ere ent an&

re)ei$e& in the ame &ataba e that ;a , e& t! re)!r& re +!n e ( U in% thi te)hni@,e ab!,t .33

,r$ey ;ere maile& ea)h &ay( Limitin% !,tb!,n& me a%e t! .33 +er &ay re&,)e& the +! ibility

!" !$erl!a&in% either the e-mail y tem !r the re ear)her e-mail in-b! (

The te)hn!l!%y !" en&in% ,r$ey by e-mail tem+t the re ear)her t! Gbat)hH e-mail

a&&re e ( St,&y . , e& thi te)hni@,e( The re ear)her )reate& a li t !" ,b?e)t an& ent the

me a%e t! all !" them in !ne me a%e( Dhile thi te)hni@,e a$e a %reat &eal !" time !n the "r!nt

en& !" the +r!?e)t# it )an )reate +r!blem ;hen ,b?e)t re +!n&( *e +ite in tr,)ti!n # !me

re +!n&ent , e& the Gre+ly t! allH ",n)ti!n !" their e-mail rea&er( Thi re ,lte& in +! ibly

ann!yin% me a%e bein% re)ei$e& by all ,r$ey ,b?e)t ( S!me ba& ;ill %enerate& "r!m the e

me a%e ;a tran "erre& t! the re ear)her(

4


14/25


9!r St,&y 4# a +r!%ram ;a ;ritten t! mer%e a li t !" e-mail a&&re e ;ith the ,r$ey

an& t! en& th! e ,r$ey $ia e-mail( Thi te)hni@,e eliminate& the +r!blem !" m,lti+le re)i+ient

"!r the ,r$ey me a%e an& it ;a m!re e""i)ient than the te)hni@,e , e& "!r St,&y ( U in% thi

te)hni@,e# ab!,t 233 ,r$ey ;ere maile& ea)h &ay(

thi ty+e !" ma mailin%( B!th the en&in% an& re)ei$in% )!m+,ter y tem may re%i ter ,)h

ma mailin% a G +amH !r ?,n5 mail( In "a)t# thi &i& ha++en( The re ear)her ,ni$er ity a))!,nt

;a tem+!rarily , +en&e& ,ntil he )!,l& a ,re the y tem mana%er that he ;a )!n&,)tin%

le%itimate re ear)h( 9,rtherm!re# !ne !" the ISP et ,+ a "ilter that +r!hibite& her "r!m

)!mm,ni)atin% ;ith any!ne ;h! ha& an e-mail a))!,nt thr!,%h that ISP( The e a&&re e ;ere

then re+la)e& in the am+lin% +!!l(

Remin&er ( A ;ith +! tal ,r$ey # a remin&er ;a ent i" a )!m+lete& ,r$ey ha& n!t

been re)ei$e& in a +e)i"ie& time( Thi a +e)t !" the meth!&!l!%y re$eal a i%ni"i)ant relati$e

a&$anta%e !" e-mail ,r$ey ( Dith +! tal ,r$ey # the re ear)her m, t all!; time "!r the ,r$ey t!

be re)ei$e& an& ret,rne& $ia +! tal er$i)e mail be"!re en&in% a remin&er( O"ten# a m!nth !r

m!re ela+ e bet;een "ir t an& e)!n& mailin% ( 9,rtherm!re# re +!n e an& remin&er mi%ht

G+a H in the mail( *,rin% the time it ta5e "!r the e tran a)ti!n t! !)),r# the ,b?e)t may "!r%et

;hether he !r he ha re +!n&e& t! the ,r$ey(

The remin&er mailin% ;a ent "r!m "i$e t! e$en &ay a"ter the !ri%inal e-mail ,r$ey ;a

ent( Be)a, e !" the +ee& ;ith ;hi)h me a%e )!,l& be ent# there ;a little &an%er !" me a%e

G+a in%H in e-mail( Dhile !me re +!n&ent in&i)ate& that they ha& n!t re)ei$e& the "ir t

me a%e# many im+ly a+!l!%i>e& "!r bein% b, y an& @,i)5ly re +!n&e& t! the remin&er( The

remin&er me a%e ent "!r the e t,&ie %enerate& "r!m .4 t! 28 +er)ent !" the t!tal re +!n e (

Re +!n&in%( S!me in&i$i&,al re+!rte& ha$in% &i""i),lty in re +!n&in% t! the ,r$ey( A"ter

e-mail e )han%e ;ith the e in&i$i&,al # it eeme& that the +rimary +r!blem ;a that they &i& n!t

2


15/25


ele)t the Gre+lyH ",n)ti!n !" their e-mail +r!%ram( They eeme& t! be ,na;are that they )!,l&

n!t ty+e re +!n e t! the ,r$ey ;hile they ;ere till in the Grea&H m!&e !" their e-mail +r!%ram(

Other &i& n!t reali>e that they )!,l& in ert their re +!n e int! the b!&y !" the ,r$ey( Thi may

eem !&& "!r +e!+le ;h! +r!bably , e e-mail ;ith !me re%,larity( De , +e)t that !ne rea !n

"!r thi +r!blem may ha$e been the n!$elty !" an e-mail ,r$ey( S!meh!;# in&i$i&,al &i& n!t

ma5e the )!nne)ti!n that thi ;a ?, t li5e any !ther e-mail J !ne m, t ele)t the re+ly ",n)ti!n

be"!re re +!n&in% an& re +!n e )an be inter +er e& ;ith the !ri%inal me a%e(

Re +!n&ent t! all three t,&ie ;ere !""ere& the !+ti!n !" +rintin% !,t the e-mail ,r$ey

an& )!m+letin% an& ret,rnin% it in thi har& )!+y $er i!n( Thi !+ti!n ;a !""ere& be)a, e !me

,b?e)t mi%ht n!t "eel )!m"!rtable ;ith the +r!)e !" )!m+letin% the ,r$ey $ia e-mail# !r mi%ht

a++re)iate the an!nymity !" en&in% the ,r$ey thr!,%h +! tal mail( 9!r all ,r$ey # le than

three +er)ent !" re +!n&ent )h! e thi !+ti!nF m! t !" the e re +!n&ent in)l,&e& their name !n

the +a+er )!+y( Thi in&i)ate& t! , that an!nymity i n!t m,)h !" an i ,e "!r the e !n-line

re ear)h ,b?e)t (

9inally# re +!n&ent eeme& t! be m,)h m!re ;illin% t! re+ly t! !+en-en&e& @,e ti!n in

the e-mail "!rmat than in tra&iti!nal +a+er ,r$ey ( The rea !n may be that m! t !" the e

re +!n&ent "in& it "a ter an& ea ier t! ty+e re +!n e than t! han& ;rite them( Al !# in !me

)a e # re +!n&ent , e& the ),t-an&-+a te ",n)ti!nality !" &i%ital )!mm,ni)ati!n t! +r!$i&e

an ;er t! !+en-en&e& @,e ti!n ( 9!r e am+le# St,&y a 5e& re +!n&ent the +,r+! e !" their

Deb ite ( Many )!+ie& their mi i!n tatement "r!m their Deb ite an& +a te& them int! the e-

mail ,r$ey( The n!$elty !" the e-mail ,r$ey may ha$e al ! em+!;ere& in&i$i&,al t! a&&

)!mmentary t! re +!n e , in% )ale ( 9!r e am+le# many re +!n&ent a&&e& )!mmentary t!

e +lain ;hy they ele)te& a )ertain re +!n e !n a @,e ti!n(

0


16/25


9in&in%

In )!m+arin% the e three t,&ie # !,r +rimary intere t ;a in h!; i ,e alien)e an& +re-

n!ti"i)ati!n mi%ht e""e)t re +!n e rate an& re +!n e +ee&( De e +e)te& that %r!,+ re)ei$in% the

m!re alient ,r$ey ;!,l& ha$e hi%her re +!n e rate an& that in&i$i&,al ;h! ha& re)ei$e& a

+re-n!ti"i)ati!n me a%e mi%ht be G+rime&H "!r the ,r$ey an& th, re +!n& m!re @,i)5ly(

e an& re +!n e rate "!r the three t,&ie (

Analy i !" Varian)e re$eale& i%ni"i)ant &i""eren)e bet;een St,&y an& St,&y 4 an& bet;een

St,&y . an& St,&y 4 9 48( 4# + (33 C( The e &i""eren)e ;ere in the e +e)te& &ire)ti!n(


17/25


i%ni"i)ant &i""eren)e bet;een St,&y . an& St,&y 4# an& n!t a i%ni"i)ant &i""eren)e bet;een St,&y

an& St,&y .(

One rea !n "!r the !me;hat l!;er re +!n e "!r St,&y . )!,l& be that the ,r$ey ;a ent t!

the Uni$er ity )!mm,nity &,rin% the ,mmer( Alth!,%h a))!,nt ;ere )he)5e& +ri!r t! en&in%

the ,r$ey t! ma5e ,re re)i+ient ha& re)ently , e& their e-mail# the ,mmer )he&,le may ha$e

re ,lte& in a &e)rea e in the "re@,en)y !" )he)5in% e-mail "!r member !" thi am+le(

*i ), i!n

E-mail ,r$ey ha$e arri$e&( It i time "!r re ear)her t! %! bey!n& im+le )!m+ari !n !"

e-mail an& +! tal mail re +!n e rate an& re +!n e +ee&( Re ear)her m, t be%in t! "!), !n why

re +!n e rate# re +!n e +ee an& !ther meth!&!l!%i)al i ,e $ary am!n% e-mail ,r$ey ( Thi

arti)le +r!$i&e an im+!rtant "ir t te+ in thi meth!&!l!%i)al &e$el!+ment by e aminin% "a)t!r

that may im+a)t !n re +!n e rate an& re +!n e +ee&(

An!ther meth!&!l!%i)al )!n i&erati!n hi%hli%hte& in thi arti)le i the im+!rtan)e an&

)hallen%e !" i&enti"yin% a++r!+riate ,ni$er e "r!m ;hi)h t! am+le( St,&ie re+!rte& in thi

/


18/25


arti)le , e& b!th !n-line an& !""-line &ire)t!rie t! )reate a am+lin% +!!l( On-line !,r)e !" e-

mail a&&re e may !""er !me ,ni@,e )hallen%e in term !" &ra;in% a ran&!m am+le( 9!r

e am+le# ele)tin% e-mail a&&re e "r!m the li t !" Deb ite +r!$i&e& by Yah!! re@,ire&

m,lti+le te+ an& te&i!, han& )!,ntin%( B,t# in !me )a e # the !n-line !,r)e )an treamline

the re ear)her ?!b( 9!r e am+le# the ran&!m li t !" e-mail a&&re e +r!$i&e& by the 9!,r

ear)h en%ine a,t!mate& the +r!)e !" ele)tin% a ran&!m am+le( A e-mail &i"", e # re ear)her

h!,l& e +erien)e little &i""i),lty in "in&in% !,r)e "r!m ;hi)h t! &ra; am+le ( The 5ey

)hallen%e ;ill be t! en ,re that th! e li t are re+re entati$e !" the ,ni$er e "r!m ;hi)h the

re ear)her ;i h t! am+le( An!ther )hallen%e ;ill be t! anti)i+ate the +er)enta%e !" ,n, able

a&&re e that may be %enerate& "r!m th! e li t (

The t,&ie re+!rte& in thi arti)le ill, trate the "a)t that many !" the tan&ar& re ear)h

te)hni@,e , e& "!r +! tal ,r$ey re ear)h )an be ea ily a++lie& t! e-mail ,r$ey ( 9!r e am+le#

remin&er mailin% )an be , e& t! b!! t the !$erall re +!n e rate( Similarly# the ann!,n)ement

me a%e ent +ri!r t! the ,r$ey a++ear t! +ee& the re +!n e time "!r the ,r$ey(

Alth!,%h alien)e !" t!+i) a++eare& t! e""e)t re +!n e rate # it i im+!rtant t! n!te that all

three t,&ie a&&re e& Internet-relate& t!+i) F th, am+lin% the Internet +!+,lati!n ;a

a++r!+riate in !r&er t! a e attit,&e an& !+ini!n !" th! e in&i$i&,al &ire)tly in$!l$e& ;ith

an& im+a)te& by the te)hn!l!%y( Dhat i n!t 5n!;n# h!;e$er# i h!; re +!n&ent ;!,l& rea)t t!

n!n-Internet relate& re ear)h( A the Internet +!+,lati!n be%in t! mirr!r the t!tal +!+,lati!n in

the Unite& State # it may be tem+tin% t! ,r$ey ),rrent , er !n all ty+e !" ,b?e)t !" intere t t!

A&$erti in% re ear)her ( De ;!,l& )a,ti!n th! e t,&yin% br!a&er i ,e that the re ,lt re+!rte&

in thi arti)le may n!t ne)e arily a++ly t! their +e)i"i) "iel& !" intere t(

S!me te)hn!l!%i)al +r!blem till nee& t! be a&&re e&( 9!r e am+le# re ear)her nee& t!

ma5e ,re they are neither a)), e& !" n!r %,ilty !" the 5in& !" G +ammin%H that )an )ri++le mail

8


19/25


er$er ( *e +ite a ,ran)e !" a)a&emi) +r!+riety in )!n&,)tin% St,&y 4# many in&i$i&,al

in&i)ate& they &i& n!t belie$e the re ear)her ;a a)t,ally an a)a&emi) re ear)her( In tea& they

belie$e& that the me a%e ha& been ent by !me!ne ;h! ;a en%a%e& in the +ra)ti)e !"

G ,%%in%H ellin% ,n&er the %,i e !" re ear)hC( Until m!re le%itimate re ear)her , e thi meth!&

an& it re)ei$e )re&ibility am!n% the Internet +,bli)# imilar it,ati!n )!,l& ari e(

A&&iti!nally# ;!r&in% nee& t! be &e$el!+e& that ;ill )lari"y the te)hni)al +r!)e !"

re +!n&in% t! an e-mail ,r$ey( Dhile +! tal mail ,r$ey are "ille& !,t , in% a ;ritin% in tr,ment#

e-mail ,r$ey all!; "!r m,lti+le re +!n e "!rmat ,)h a in ertin% the ),r !r int! the &e ire&

+!int an& ty+in% in an an ;er# an ;erin% $ia a G"!rm#H an& +r!$i&in% an ;er in a e+arate

me a%e( All "!rmat may n!t ;!r5 !n all y tem "!r e am+le# !me AOL member )!,l& n!t

re +!n& by in ertin% the ),r !r at a +e)i"i) +!int in the te tC(

A +!tential limitati!n t! thi t,&y i that the &e%ree !" +er !nali>ati!n $arie& !me;hat

am!n% the t,&ie e$en th!,%h all ,r$ey ;ere +er !nali>e& t! !me &e%ree( In St,&y #

re +!n&ent ;ere re@,e te& t! an ;er @,e ti!n re%ar&in% a +e)i"i) Deb ite that they ha&

&e$el!+e an& the +e)i"i) URL ;a +r!$i&e& t! ea)h +!tential re +!n&ent ( Thi may ha$e

in)rea e& b!th +er !nali>ati!n an& i ,e alien)e( St,&ie . an& 4 &i& n!t ha$e thi ty+e !"

+er !nali>ati!n( A&&iti!nally# St,&y . ;a maile& , in% a bat)h mail +r!%ram( Dith thi meth!

the re)i+ient name i in)l,&e& in the t,&y# yet the mail +r!%ram may in&i)ate that he !r he ;a

+art !" a mailin% Gli t(H Dhether thi ty+e !" mailin% te)hni@,e e""e)t re +!n e rate a either a

+rimary !r m!&eratin% e""e)t h!,l& be re ear)he& ",rther(

9in&in% relate& t! e-mail ,r$ey h!,l& en)!,ra%e re ear)her ( U in% e-mail# re ear)her

)an @,i)5ly a e )!n ,mer !+ini!n( 9,rther re ear)h h!,l& "!), !n ;ay t! in)rea e re +!n e

rate an& )a+itali>e !n !ne !" the m! t e$i&ent bene"it !" e-mail ,r$ey : the "a t re +!n e +ee&(

E$en am!n% G l!;erH re +!n&ent %r!,+ # m! t ,r$ey are an ;ere& in le than !ne ;!r5 ;ee5(

1


20/25


In )!n)l, i!n# e-mail ,r$ey !""er an e )itin% !++!rt,nity "!r re ear)her : )! t are l!;#

re +!n e rate are %!! an& re +!n e time are @,i)5( A re ear)her a++ly the e te)hni@,e t!

many ty+e !" a&$erti in% @,e ti!n # the re ear)h )!mm,nity )an ;!r5 t!%ether t! re"ine e-mail

,r$ey meth!&!l!%y(

.3


21/25


Table ( S,mmary !" S,r$ey Re ear)h Meth!& U in% E-mail

A,th!r Sam+le S,r$ey T!+i) Meth!& Sam+leSi>e

T!talRe +!n e

Re +!n eRate

Re +!n eTime

&ay CKie ler S+r!,ll

187CEm+l!yee !" a9!rt,ne 033=!m+any

=!r+!rate=!mm,ni)ati!n

MailE-MailT!tal

00

.43

// 87

74

7//0

3(81(7

Par5er 11.C Em+l!yee !"AT T

Internal=!mm,ni)ati!n

MailE-MailT!tal

/3 /3

23

./28/0

4878

NA NA

S)h,l&t T!tten112C

Mar5etin% MISPr!"e !r USC

Share;are =!+yin% MailE-MailT!tal

.33

. 82 8

4 2.

00

07(01(4

NA NA

Mehta Si$a&a110C

U enet U er Internet=!mm,ni)ati!n

MailE-MailT!tal

4318.

21

/4 11./.

07(002(4

NA NA

T e# et al 110C Uni$er ityP!+,lati!n

B, ine Ethi) MailE-MailT!tal

.33

.33233

02.

77

./7

1(/18(31

Ba)hman# El"rin5 Va>>ana 117C

B, ine S)h!!l*ean

T M MailE-MailT!tal

..2

..2228

2//

.72

70(70.(0

( 82(78

Deible an& Dalla)e118C

MIS Pr!"e !rUSC

Internet U e Mail9aE-MailDeb 9!rmT!tal

.33

.33

.33

.33833

/30328

0...3

40(/.3(1.1(84.(/

.(18(87(/(2

S)hae"er an& *illman118C

Uni$er ity 9a),lty Un5n!;n MailE-MailT!tal

..7

..720.

434

.7.

0/(008(3

2(411( 7

*i""eren)e n!t i%ni"i)ant

.


22/25


Table .( S,mmary !" the Three St,&ie

St,&y St,&y . St,&y 4

Sam+lin% "rame =reat!r !" health-relate& Deb ite

9a),lty# ta"" an&t,&ent at a ma?!r

re ear)h ,ni$er ity

In&i$i&,al ;ith +er !nal e-mail

a))!,nt

St,&y t!+i) Val,e !" ite)reat!r F ite +,r+! e an&

",n&in%

Attit,&e t!;ar&!n-line +ri$a)y

Attit,&e an& beha$i!r

a !)iate& ;ith !n-line +ri$a)y

Time 9rame 'an,ary-9ebr,ary11/

May-',ly# 117 N!$ember 11/-'an,ary 118

N,mber !" @,e ti!n 40 20 47

N,mber !" )!m+,ter )reenthe ,b?e)t m, t re$ie;

, in% PineC

7 .3 8

Mailin% meth!& In&i$i&,al Bat)h Mer%e +r!%ram

S!li)itati!n ent be"!re,r$ey

N! Ye Ye

Remin&er ent t! n!n-re +!n&er a"ter h!; many&ay

/ 7 0

Per)ent !" t!tal re +!n ethat )ame a"ter the remin&er

28 .0 .4

Per)ent re +!n&in% , in% +! tal rather than e-mail

.(/ (8 (/

..


23/25


Table 4( S,r$ey Re +!n e Rate

Vali& Sam+le

Si>e

T!tal Re +!n e Re +!n e Rate

St,&y J Deb *e$el!+er 842 410 2/(2

St,&y . J Uni$er ity 083 ./2 2/(.

St,&y 4 JIn&i$i&,al ;ithPer !nal e-mail a))!,nt

4#/.2 810 .2(3

.4


24/25


Re"eren)e

Arm tr!n% '(S( an& E( '( L, 5( GRet,rn P! ta%e in Mail S,r$ey # A Meta-analy i (H P,bli)O+ini!n ,arterly 0 18/C: .44-.28(

Ba)hmann# *(# '( El"rin5 an& ( Va>>ana( GTra)5in% the Pr!%re !" E-mail Ver , Snail Mail(HMar5etin% Re ear)h 8 117C: 4 -40(

*e+artment !" =!mmer)e( GThe Emer%in% *i%ital E)!n!my(H 118C( A$ailable:htt+: ;;;(e)!mmer)e(%!$ &anintr!(htm

*illman# *( A( GThe *e i%n an& A&mini trati!n !" E-mail S,r$ey (H Ann,al Re$ie; !" S!)i!l!%y/ 11 C: ..0-21(

*illman# *( A( Mail an& Tele+h!ne S,r$ey : The T!tal *e i%n Meth!&( Ne; Y!r5: '!hn Dileyan& S!n # 1/8(

*,n)an# D( '( GMail ,e ti!nnaire in S,r$ey Re ear)h: A Re$ie; !" Re +!n e In&,)ementTe)hni@,e (H '!,rnal !" Mana%ement 0# 1/1C: 41-00(

9! # R(# M(R( =ra 5# an& '( Kim( GMail S,r$ey Re +!n e Rate : a Meta-Analy i !" Sele)te&Te)hni@,e "!r In&,)in% Re +!n e(H P,bli) O+ini!n ,arterly 0. 188C: 27/-21 ((

!y&er# '( =( G9,rther E$i&en)e !n 9a)t!r A""e)tin% Re +!n e Rate t! Maile& ,e ti!nnaire (HAmeri)an S!)i!l!%i)al Re$ie; 2/ 18.C: 003-004(


25/25


Par5er# L( G=!lle)tin% *ata the E-mail Day(H Trainin% an& *e$el!+ment ',ly 11.C: 0.-02(

R!ber !n# M(T( an& E( S,n& tr!m( G ,e ti!nnaire *e i%n# Ret,rn Rate # an& Re +!n e9a$!rablene in an Em+l!yee Attit,&e ,e ti!nnaire(H '!,rnal !" A++lie& P y)h!l!%y /0 113C:402-40/(

S)hae"er# *( R( an& *( A( *illman( G*e$el!+ment !" a Stan&ar& Email Meth!&!l!%y: Re ,lt !"an E +eriment( P,bli) O+ini!n ,arterly 4# 7. 118C: 4/8-413(

S)h,l&t# B(A( an& '( T!tten( GEle)tr!ni) Mail $ ( Mail S,r$ey Re +!n e Rate (H Mar5etin%Re ear)h 112C: -/(

Steele# T( '(# D( L( S)h;en&i% an& '( A( Kil+atri)5( G*,+li)ate Re +!n e t! M,lti+le S,r$eyMailin% : A Pr!blem H '!,rnal !" A&$erti in% Re ear)h Mar)h A+ril 11.C: .7-42(

Tayl!r# S# an& P( Lynn( GThe E""e)t !" a Preliminary N!ti"i)ati!n Letter !n Re +!n e t! a P! talS,r$ey !" Y!,n% Pe!+le(H '!,rnal !" the Mar5et Re ear)h S!)iety .# 23 118C: 70- /8(

T e# A(# K(=( T e# =(

Yammarin!# 9('(# S( S5inner an& T( L( =hil&er ( GUn&er tan&in% Mail S,r$ey Re +!n eBeha$i!r(H P,bli) O+ini!n ,arterly 00 11 C: 7 4-741(

Y,# '( an&

Documents

Research About Survey via E-mail