Improving Response RatesLessons from Physician Surveys

PMRS Ottawa Chapter February 26, 2004

Presentation Overview

Survey Response Rates: The state of the art Particularities of Physician Surveys Response rate boosting tactics: What works

and what does not. On the use of Monetary Incentives in Physician

and Consumer Surveys Q & A period

Current response rates

Academic Surveys published between 1961 and 1977: 71%

Academic surveys published in 1991: 54% Academic surveys published between 1986 and

1995, sample size over 1,000 respondents: 52% Commercial/marketing physician surveys

(2002): 20% RETICULUM surveys: 12% to 66%

Current response rates

Surveys of executives, published in 1991: 21% PMRS Members surveys: 15.7%(1997);11.3%

(2000); One-time telephone surveys: 16% (1997); 13%

(2002)

Physician Surveys Particularities

More homogenous populations

Highly-regulated professionals

Better sampling frames

Better record-keeping

Physician Surveys Particularities

Highly-solicited respondents Highly-educated respondents ‘Well-connected’ respondents

Physician Surveys Particularities

Surrounded by ‘tough’ gatekeepers

‘Addicted’ to monetary incentives

Tactics that boost response by 50% or more

Monetary Incentives

Multiple contacts & multiple contact modes

Monetary Incentives

(Gallagher, 2001)1st contact by mail, no incentive: 11%2nd contact by phone, no incentive: 22% (cumul)3rd contact by courier, $20 incentive: 57% (cumul)

(Malin, 2000)1st mailing, no incentive: 17%2nd mailing, no incentive: 13%3rd mailing, $50 incentive: 66%Cumulative response rate: 76%

Multiple Contacts & Contact Modes

Typical response rates after multiple mailings:

1st mailing: X%

2nd mailing: X/2%

3rd mailing: X/4% (CDC, 1997):

1st contact by First Class mail: 60%

2nd contact by Fedex: 72% (cumul)

3rd contact by phone: 96% (cumul)

Tactics that boost response by a few % points

Pre-notification by phone Personalization Advertising the survey Choice of sponsors Shortening the questionnaire Instituting a draw

Pre-notification by phone

(Osborn, 1996)

No Pre-notification: 64%

Pre-notification: 77%

(Ward, 1994)

No Pre-notification: 69%

Pre-notification: 84%

Personalization

First Class mail Commemorative stamps Stamped return envelope Name & address printed on the envelope Personalized salutation Full date on Cover Letter Handwritten signature Handwritten note

Personalization

(Maheux, 1989)

Handwritten ‘thank you’ note: 30%

No ‘thank you’ note: 22% (Streiff, 1999)

Stamped return envelope: 38%

Business-reply envelope: 32%

Choice of sponsor

(Asch, 1994)

Veteran Affairs return address pulled 20% more than a Hospital Department of Medicine

(RETICULUM, 2000)

A joint study with IMS Health, Royal College, College of Family Physicians: 22%

IMS Health alone: 12%

Tactics that don’t boost response

Pre-notification by mail Offering non-monetary incentives (pens,

mouse pads, candy, booklets, software..) Mailing surveys on a specific day of the week Promising anonymity Gimmicks

On the use of Monetary Incentives

Even symbolic sums will boost response (Everett,1997)

$0: 45%

$1: 63% (one-dollar bill included in mailing) (Donaldson, 1999)

$0: 46%

$5: 58% (five-dollar cheque included)

On the use of Monetary Incentives

Larger incentives, Higher response rates (Asch, 1998)

$2 incentive: 46%$5 incentive: 63%

(Gunn, 1981)$0 incentive: 58%$25 incentive: 69%$50 incentive: 77%

On the use of Monetary Incentives

Larger incentives, Higher response rates:UP TO A POINT (VanGeest, 2001)

$5 incentive: 60% ;$10 incentive: 68%$20 incentive: 67%

(RETICULUM/ IMS Health, 2000)$25 incentive: 22%; $50 incentive: 34%$75 incentive: 36%

On the use of Monetary Incentives

Pre-paid incentives outperform

Post-paid incentives (Berry, 1987)

$20 incentive, pre-paid: 78%

$20 incentive, post-paid: 66%

Monetary Incentives in Lay Surveys

Sparse data Controversial practice Banned in certain jurisdictions

Monetary Incentives in Lay Surveys

(James & Bolstein, 1992)

$0: 52%

$2: 64% (Dillman and al., 1999) in 5 different studies

$2 incentive: boosted response by 19 to 31%

Monetary Incentives in Lay Surveys

Pre-paid incentives will outperform post-paid incentives (Johnson & McLaughlin, 1990):

$5 pre-paid: 83%

$10 post-paid: 72% (James & Bolstein, 1992): survey of small contractors

$1 pre-paid: 64%

$5 pre-paid: 72%

$50 post-paid: 57%

On the use of Monetary Incentives

Why and how do they work???

On the use of Monetary Incentives

Respondent appreciated, not taken for granted Value-creating Attention grabbing: Secretary Attention grabbing: Physician Pre-paid incentives: create trust

On the use of Monetary Incentives

The Pitfalls: Point of no-return Cost Fraud Ethical Issues

On the use of Monetary Incentives

Pre-paid incentives (Gallagher, 2001)

46% replied

3% declined and returned the 20-dollar pre-paid incentive

51% declined, but pocketed the 20-dollar pre-paid incentive

PMRS Ottawa Chapter

Thank you very much Q & A (in both official languages!)