1 Impact of Changes in the Telephone Environment On RDD Telephone Surveys Mary Cay Murray Abt...

1

Impact of Changes in the Telephone Environment

On RDD Telephone Surveys

Mary Cay MurrayAbt Associates Inc

Erin Foster Abt Associates Inc

Jessica CardoniAbt Associates Inc

Chris BeckerAbt Associates Inc

Paul BuckleyConsultant

Marcie CynamonNational Center for Health Statistics

2

Changes in the Telephone Environment

Topics:

Out-of-scope lines

Answering machines

Privacy manager devices

Do-Not-Call lists

Database: The National Immunization Survey, (Q1/98-Q4/02)

3

The National Immunization Survey (NIS)Data collection conducted quarterly since April of 1994

Sponsored by the National Immunization Program and the National Center for Health Statistics, both of the Centers for Disease Control and Prevention

National study—data collection in 78 non-overlapping Immunization Action Plan (IAP) Areas:

50 states, 27 metropolitan areas, and the District of Columbia

Yearly, 10 million telephone calls made to identify households with 35,000 age-eligible children

4

NIS, Continued

Measures vaccination coverage of children between the ages of 19 and 35 months

3,000,000 sample telephone numbers drawn per year

List-assisted design

Processed through GENESYS-IDplus® before CATI screening

1,000,000 households contacted per year

5

Growth in Out-of-Scope Lines

Growth across Quarters

Non-working Numbers by Census Division

6

NIS: Estimated Out-of-Scope Percentages by Quarter

0%

10%

20%

30%

40%

50%

Q1/1998

Q2/1998

Q3/1998

Q4/1998

Q1/1999

Q2/1999

Q3/1999

Q4/1999

Q1/2000

Q2/2000

Q3/2000

Q4/2000

Q1/2001

Q2/2001

Q3/2001

Q4/2001

Q1/2002

Q2/2002

Q3/2002

Q4/2002

Estimated Out-of-Scope Percentages by Quarter

Total Non-w orking Total Business Total Fax/Modem

7

Census Divisions

8

NIS: Non-working Numbers by Census Division

Percent Non-working by Census Division

25%

30%

35%

40%

45%

50%

Quarter

Census 1 New England

Census 2 Middle Atlantic

Census 3 East North Central

Census 4 West North Central

Census 5 South Atlantic

Census 6 East South Central

Census 7 West South Central

Census 8 Mountain

Census 9 Pacif ic

9

NIS: Non-working Numbers by Census Division – Q4/00 and Q4/02

Table 1. Percent Non-working by Census DivisionQ4/2000 Q4/2002

New England 31.21% 34.41%Middle Atlantic 29.55% 31.89%East North Central 37.34% 40.73%West North Central 40.45% 45.09%South Atlantic 31.60% 34.12%East South Central 31.45% 36.13%West South Central 35.13% 40.23%Mountain 36.44% 39.95%Pacific 33.14% 35.18%

10

NIS: Answering Machines

Next, we will examine the trends in the prevalence and use of answering machines.

11

NIS: Percent Ever Answering Machine by Quarter

20.4%19.0%

0%

5%

10%

15%

20%

25%

Q1/

2000

Q2/

2000

Q3/

2000

Q4/

2000

Q1/

2001

Q2/

2001

Q3/

2001

Q4/

2001

Q1/

2002

Q2/

2002

Q3/

2002

Q4/

2002

Quarter

Ever Answering Machine

12

NIS: Percent Finalized Answering Machines by Quarter

1.03%

2.82%

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

Q1/1998 Q3/1998 Q1/1999 Q3/1999 Q1/2000 Q3/2000 Q1/2001 Q3/2001 Q1/2002 Q3/2002

Percent Finalized Answering Machines by Quarter

13

NIS: Percent Finalized Answering Machines by Census Division by Quarter

Percent Finalized Answering Machines by Census Division

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

3.5%

Q1/20

00

Q2/20

00

Q3/20

00

Q4/20

00

Q1/20

01

Q2/20

01

Q3/20

01

Q4/20

01

Q1/20

02

Q2/20

02

Q3/20

02

Q4/20

02

Census 1 NewEngland

Census 2 MiddleAtlantic

Census 3 East NorthCentral

Census 4 WestNorth Central

Census 5 SouthAtlantic

Census 6 East SouthCentral

Census 7 WestSouth Central

Census 8 Mountain

Census 9 Pacif ic

14

NIS: Percent Finalized Answering Machines by Census Division – Q1/00 and Q4/02

Table 2. Percent Finalized Answering Machines by Census DivisionQ1/2000 Q4/2002

New England 1.67% 3.04%Middle Atlantic 1.56% 3.06%East North Central 1.32% 2.66%West North Central 0.88% 1.66%South Atlantic 1.72% 3.57%East South Central 1.10% 3.05%West South Central 1.22% 2.64%Mountain 1.23% 2.22%Pacific 1.58% 2.65%

15

Privacy Managers in the NIS

What do we mean by Privacy Manager?

A call-screening device that works with caller-ID to manage all incoming calls.

16

Coding Rules for Privacy Manager Calls

If the line rang with no answer by the household, but the interviewer identified the study when asked to do so by the Privacy Manager, the case is coded as PM ring-no-answer;

If the Privacy Manager denied the call, the case is coded as a PM hang-up-during-introduction;

If the call is sent to an answering machine, the case is coded as answering machine.

17

NIS: Percent Identified Privacy Manager by Quarter

0.79%

1.73%

0.00%

0.25%

0.50%

0.75%

1.00%

1.25%

1.50%

1.75%

Q201 Q301 Q401 Q102 Q202 Q302 Q402

Percent Identified Privacy Manager

18

NIS: Identified Privacy Manager by Census Division by Quarter

Identified Privacy Manager by Census Division

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

Q2/01 Q3/01 Q4/01 Q1/02 Q2/02 Q3/02 Q4/02

Quarter

Census 1 NewEngland

Census 2 MiddleAtlantic

Census 3 EastNorth Central

Census 4 WestNorth Central

Census 5 SouthAtlantic

Census 6 EastSouth Central

Census 7 WestSouth Central

Census 8Mountain

Census 9 Pacif ic

19

NIS: Identified Privacy Manager by Census Division – Q2/01 and Q4/02

Table 3. Percent Identified Privacy Manager by Census DivisionQ2/2001 Q4/2002

New England 0.15% 1.01%Middle Atlantic 0.21% 1.26%East North Central 1.49% 2.08%West North Central 0.68% 0.78%South Atlantic 0.85% 2.41%East South Central 1.33% 2.82%West South Central 0.70% 2.29%Mountain 0.95% 1.11%Pacific 0.41% 0.89%

20

NIS: Privacy Manager by Census Division, Log

Log of Identified Privacy Manager by Census Division

0.1%

1.0%

10.0%

Q2/01 Q3/01 Q4/01 Q1/02 Q2/02 Q3/02 Q4/02

Quarter

Census 1 NewEngland

Census 2 MiddleAtlantic

Census 3 EastNorth Central

Census 4 WestNorth Central

Census 5 SouthAtlantic

Census 6 EastSouth Central

Census 7 WestSouth Central

Census 8Mountain

Census 9 Pacif ic

21

NIS: Identified Privacy Manager OutcomesOutcomes of Identified Privacy Manager Cases

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Q201 Q301 Q401 Q102 Q202 Q302 Q402

Household Likely household Answering Machine

Business Non-contact Non-working

22

State Do-Not-Call Lists

These lists do not apply to surveys such as the NIS

Telephone numbers on these lists are likely to be households

Identified by Marketing Systems Group

23

NIS: State Do-Not-Call Lists

2.4% 2.4%

3.1%

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

3.5%

Q2/2002 Q3/2002 Q4/2002

Percent Identified as State Do-Not-Call List byMarketing Systems Group's Genesys ID Plus

24

NIS: State Do-Not-Call Lists by Census Division – Q2/02 and Q4/02

Table 4. State Do-Not-Call Lists by Census DivisionQ2/2002 Q4/2002

New England 3.21% 3.17%Middle Atlantic 3.23% 5.52%East North Central 2.87% 4.31%West North Central 3.47% 4.19%South Atlantic 1.28% 1.42%East South Central 6.15% 8.89%West South Central 0.67% 0.70%Mountain 1.13% 2.69%Pacific 1.06% 1.11%

25

NIS: Final Outcomes of Do-Not-Call List Cases

Final Outcomes of State Do-Not-Call List Cases

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

Q2/02 Q3/02 Q4/02

Household Business Non-working Likely household Answering Machine Non-contact

26

NIS Do-Not-Call List

Maintained cumulatively since 1998

Telephone numbers can be sampled again across quarters in the NIS

27

NIS Do-Not-Call List: Percent of Lines by Quarter

0.48%

0.97%

0.0%

0.1%

0.2%

0.3%

0.4%

0.5%

0.6%

0.7%

0.8%

0.9%

1.0%

Q1/

1999

Q2/

1999

Q3/

1999

Q4/

1999

Q1/

2000

Q2/

2000

Q3/

2000

Q4/

2000

Q1/

2001

Q2/

2001

Q3/

2001

Q4/

2001

Q1/

2002

Q2/

2002

Q3/

2002

Q4/

2002

Percent of Total Lines Requesting No Call List

28

Conclusions

Increase in non-working numbers leads to fewer household lines in samples

Answering Machines continue to be used to screen calls and increasingly to avoid participating in surveys

Privacy Managers are growing, but still a small portion of total sample

Cases on state do-not-call lists are generally cooperative

“Take me off your list” is an easy way to opt out of a survey

Abt Associates Inc.

mary_cay_murray@abtassoc.com