1

Patient Experience Surveys—Spreading Their Reach

by Robert Krughoff and Paul Kallaur

Consumers’ CHECKBOOK/Center for the Study of Services

Consumer-Purchaser Disclosure Project

July 12, 2007

2

Cost of Physician Surveys• Survey Administration

– C/G CAHPS mail-mode protocol is about $220 per physician (assuming need for 40 responses and get 36% response rate).

– $110 million to get results on 500,000 physicians annually.

– Requires surveying about 56 million patients.

• Reporting Results– Quality improvement

– Public reporting

3

Key Factors Affecting Costs• How often survey is done—every three years vs every year cuts the annual national cost from $110

million to $37 million, the per-citizen annual cost to about 12 cents.

• Whether just a subset of specialty types is included—for example, PCPs, Ob/Gyns, cardiologists, gastroenterologists.

• Number of completed surveys required per physician—for example, moving from average of 40 completes per physician to 32 completes per physician cuts cost by 20% and still may allow statistically significant distinctions among physicians in some markets on some dimensions.

• Protocol options—– Nonprofit mail rates possibly cuts 20%– First wave by e-mail might save in future if—plans/groups/docs can get e-mail addresses, spam

standards and blocks can be accommodated, privacy issues with shared or corporate e-mail addresses can be addressed

• How many plans share in cost for a physician• What other organizations contribute to cost. For example—

– Specialty boards have expressed interest in contributing because they can use survey results in Maintenance of Certification programs

– Malpractice insurers might pay for the data for use in underwriting – Medicare and Medicaid might eventually contribute

4

Alternate Modes of Administration Don’t Pay Off

• Passive Web survey on plan website– Plans trying this have gotten tiny response – Concerns about bias– Worth continuing to test and find ways to promote, but not

currently promising

• Hand-outs of questionnaires or survey invitations in physicians’ offices– Has been used by specialty boards, but not where scores counted– Distribution can be cheaper to the study sponsor than mail, but

may impose hidden costs on the practice, and on the sponsor to get practices to participate and to audit

– Might be difficult for a plan or other sponsor to implement (and audit) on a large scale—especially if physicians are resistant

– Mode/physician interaction effects observed in tests raise questions about manipulation by physicians. Will results be credible?

5

Key Question: Who Pays for the Survey– The user(s) of the information—

• Consumers (through information publishers)• Employers• Plans• Medical groups• Government payers• Government health agencies• Specialty boards• Malpractice insurers• Multi-stakeholder consortiums

– Physicians, practices, or medical groups being evaluated—for intrinsic self-improvement motives, to earn recognition/rewards for doing survey or performing well (in PVRP, P4P, recognition program, certification programs), or because purchasers or governments simply require it

– Either way, consumers/the public pays indirectly

6

Implementation Models for CAHPS Clinician & Group Survey• Regional collaboratives

– BQI markets– Aligning Forces markets

• Accreditation/certification– American Board of Medical Specialties

• Independent efforts– Health plans– Medical groups

• National health plan consortium– CSS initiative

7

Collaborative Model: Health Plan Driven• Major health plans contract to contribute to survey costs

(formula that takes into account number of docs on which they want data, number of members who’ll benefit from data, a basic pay-to-play element)

• Plans provide survey sampling frame of physicians’ patients (pooled across plans) from claims data

• Contributing plans get rights to resulting data for their use for provider directories, P4P, recognition programs, etc.

• Medical specialty boards get data on docs who are up for maintenance of certification and boards or docs contribute to survey costs

• Medical groups and hospitals can buy rights to data from collaborative (medical groups might be invited or required by plans to participate in initial survey costs)

• Information publishers (WebMD, Consumer Reports, CHECKBOOK, Revolution Health, Healthgrades, etc.) will be able to purchase rights to survey results for publication directly to consumers

• Other users such as malpractice insurers will be able to purchase survey results

8

Collaborative Model: Physician Driven• Physicians or medical groups volunteer and pay for survey• Physicians/groups own the data and decide to whom to

release it (option 1: must agree to make public before collection; option 2: decide after seeing their data)

• Plans collaborate to provide survey sample (or medical groups/physicians provide survey sample if plans can’t, and plans selectively audit)

• Various parties collaborate to create incentives for voluntary participation—plans and other payers (P4P, PVRP, recognition programs, etc. using Bridges to Excellence, NCQA Medical Home, and similar approaches), specialty boards for maint. of certif., information publishers, malpractice insurers, etc.)

• Collaborative role same as in previous model for independent survey implementation, reporting standards, etc.

9

To Move Toward Goal of Widespread Surveys of Patients About Physicians

• CSS/CHECKBOOK and consumer/purchaser leaders need to—– Try to arrange for plans that collaborate on patient surveys to be scored higher by

plan evaluation tools like NBCH’s eValuate, NCQA’s Quality Plus, and Leapfrog’s scorecards.

– Work with Bridges to Excellence, NCQA’s Medical Home effort, Medicare measurement programs (including PVRP), and other programs to ensure that P4P programs and network designs reward physicians who participate and score well in patient surveys.

– Work with specialty board leadership on integrating patient survey results into Maintenance of Certification and quality improvement programs.

– Work with malpractice insurers and their associations to assess the usefulness of a physician’s patient survey results as an underwriting element.

– Work with AQA leadership and CMS/AHRQ to move the collaborative patient survey approach forward on the agenda of BQIPs and Value Exchange pilots—and possibly to make collaborative patient surveys the initial organizing catalyst for coalitions in some communities.

– Recruit community coalitions to move forward the collaborative patient survey approach in their communities.

10

We First Published Consumer Survey Ratings of Physicians in 1980 and Hope Progress Will Be

Faster in the Next 27 Years

11

Appendix

12

• Sampling at beginning, and analysis and scoring at back end, are done by collaborative

• Survey fielding is done by contractor or contractors competitively chosen by committee of plans, specialty boards, and other collaborative participants

• Plans pay for survey only every three years• Physicians or medical groups who want to be surveyed more

often can pay for those surveys—with plans providing sampling frame and collaborative independently sampling, fielding, and analyzing results

• Physicians not affiliated with participating plans (or with too little sample in those plans) can arrange for surveys by providing sampling frame through medical groups, if so affiliated, or directly—so collaborative can sample, field, and analyze results (collaborative with help of affiliated plans will attempt selectively to audit sampling frame for completeness)

Collaborative Model: Health Plan Driven(more details)

13

• Work with community coalitions (BQIPs, Value Exchanges, etc.) where possible—and seek to provide an initial organizing principle for coalitions where they do not already exist

• Welcome participation from government agencies and purchasers, including Medicare, and possibly provide a vehicle for incorporation of patient experience surveys into PVRP

• Use C/G CAHPS survey questionnaires and protocols developed by AHRQ as approved by NQF

• Allow plans or others who have license to use survey results to have flexibility in reporting but not to change underlying scores calculated by collaborative (for example, by changing case-mix adjustment method)

• Require all reports of results to adhere to well-accepted reporting principles (for example, AQA’s principles for public and provider reporting)

• Allow individual physicians an opportunity to see their results before public release

Collaborative Model: Health Plan Driven (more details)

14

What CHECKBOOK/CSS Will Be Doing in Coming Months

• Continue to revise the description of the collaborative models based on feedback from advisory committee and others, including seeking to share costs and capture some survey responses more efficiently

• Seek several communities that are interested in being sites for pilot surveys for the collaborative—both models

• Recruit health plans, specialty boards, medical groups, foundations, and others to participate in, and contribute to, the pilot projects

• Hope to launch pilot projects this fall

15

Advisory Committee to CHECKBOOK/CSS Collaborative• Reed Tuckson, MD, Senior Vice President for

Consumer Health and Medical Care Advancement, UnitedHealthcare

• Dick Salmon, MD, PhD, Vice President and National Medical Executive, CIGNA

• Paul Thompson, Director, National Cost & Quality Transparency Initiatives, CIGNA

• Chuck Cutler, MD, M.S., National Medical Director, Aetna

• Thomas James, MD, Chief Medical Officer, Humana, Kentucky

• Arnold Milstein, MD, Medical Director of Pacific Business Group on Health and the National Health Care Thought Leader at William M. Mercer Co.

• Andy Webber, President and CEO, National Business Coalition on Health

• Francois DeBrantes, National Coordinator, Bridges to Excellence

• Debra Ness, CEO, National Partnership for Women and Families

• Joyce Dubow, Associate Director, AARP Public Policy Institute

• Jim Guest, President and CEO, Consumer Reports/Consumers Union

• Melinda Karp, Director of Programs, Massachusetts Health Quality Partners

• Cary Sennett, MD, Senior Vice President for Research, American Board of Internal Medicine

• F. Daniel Duffy, MD, Executive Vice President, American Board of Internal Medicine

• Steve Miller, MD, MPH, President, American Board of Medical Specialties

• Amy Mosser, Vice President, American Board of Medical Specialties

• Carmella Bocchino, Executive Vice President, Americas Health Insurance Plans

• Charles Darby, CAHPS Project Officer, U.S. Agency for Healthcare Research and Quality

• Bernard Rosof, MD, Co-chair, AMA Physician Consortium for Performance Improvement and Senior Vice President, North Shore-Long Island Jewish Health System

• David Stumpf, MD, Medical Director, UnitedHealthcare Clinical Operations

• Gregory Pawlson, MD, MPH, Executive Vice President, National Committee for Quality Assurance

• Carol Cronin, consumer information consultant • Michael Barr, MD, Vice President, Practice Advocacy

and Improvement, American College of Physicians • Paul V. Miles, MD, Vice President and Director of

Quality Improvement, American Board of Pediatrics • Peter Hayes, Health Benefits Strategist, Hannaford

Bros. • Lee Tiedrich, Partner, Covington & Burling, LLP

16

Benefits of Multi-User Collaboration• CHECKBOOK/CSS has been working to build a

collaborative of plans, specialty boards, and others (see advisory committee list at end of presentation)

• Docs’ scores don’t seem to depend much on plan or group—so one score per doc may be able to be used by all

• Avoid duplicative cost of survey set-up and fielding• Get adequate sample sizes per doc• Minimize survey respondent burden• Produce consistent results/scores—simplifying for

consumers and providers• Insulate against possible physician resistance• Be a better candidate for public/government support• Individual organizations can still distinguish themselves by

how they use the survey results