0000605-201306040-00003.pdf

8/14/2019 0000605-201306040-00003.pdf

1/13

Meeting Meaningful Use Criteria and Managing Patient PopulationsA National Survey of Practicing PhysiciansCatherine M. DesRoches, DrPH; Anne-Marie Audet, MD; Michael Painter, MD; and Karen Donelan, ScD

Background: Meaningful use, as defined by the Centers for Medi-care & Medicaid Services, will require the aggregation of patientdata to enable population assessment. Little is known about theproportion of physicians who are able to meet meaningful usecriteria or their use of electronic health records (EHRs) to managepatient populations.

Objective: To evaluate physicians reports of EHR adoption andease of use and their ability to use EHRs for patient panelmanagement.

Design: National mailed survey of practicing physicians (responserate of 60%).

Setting: Late 2011 and early 2012.

Participants: 1820 primary care physicians and specialists in office-

based practices.Measurements: Proportion of physicians who have a basic EHRand meet meaningful use criteria and ease of use of computerizedsystems designed for patient population management tasks.

Results: A total of 43.5% of physicians reported having a basicEHR, and 9.8% met meaningful use criteria. Computerized systems

for managing patient populations were not widespread; fewer thanone half of respondents reported the presence of computerizedsystems for any of the patient population management tasks in-cluded in the survey. Physicians with such functionalities reportedthat these systems varied in ease of use. Physicians with an EHRthat met meaningful use criteria were significantly more likely thanthose not meeting the standard to rate panel management tasks aseasy.

Limitation: Ease-of-use measures are subjective.

Conclusion: Few physicians could meet meaningful use criteria inearly 2012 and using computerized systems for the panel manage-ment tasks was difficult. Results support the growing evidence thatusing the basic data input capabilities of an EHR does not translateinto the greater opportunity that these technologies promise.

Primary Funding Source: Commonwealth Fund and Robert WoodJohnson Foundation.

Ann Intern Med. 2013;158:791-799. www.annals.orgFor author affiliations, see end of text.

The Health Information Technology for Economic andClinical Health Act of 2009 authorized the Centers forMedicare & Medicaid Services (CMS) to spend billions of dollars in the form of incentive payments to providers toencourage the adoption and meaningful use of electronichealth records (EHRs) (1). To qualify for these payments,physicians must attest to meeting a series of criteria, cate-gorized as an escalator that moves physicians toward thegoal of providing high-quality, efcient, patient-centeredcare (2, 3). A basic assumption underlying the meaningfuluse criteria is that achieving these goals will require phy-sicians to use the data in their EHR to measure and assessthe care they provide to their patient populations (4).

The meaningful use program is being implemented instages. To meet the stage 1 criteria, eligible providers mustattest to using an EHR to electronically capture healthinformation in a coded format, track key clinical condi-tions and communicate that information for care coordi-nation purposes, facilitate disease and medication manage-ment, and report clinical quality measures and publichealth information. To meet stage 2 criteria, nalized inSeptember 2012, physicians must go beyond simple attes-tation and demonstrate that they are using their EHRs tosupport individualized care for patients and patient panelmanagement tasks, such as using clinically relevant infor-mation to identify patients who should receive remindersfor preventive/follow-up care (5).

Measures of EHR adoption typically focus on func-tionalities of systems and the ability of physicians to accessand store information at the point of care. Using an EHR in a way that may result in higher-quality, more efcientcare, however, will probably require physicians to aggregateindividual-patient data to enable population assessmentand management. This activity requires that cliniciansknow how to query and analyze data and use the informa-tion to change practice.

Current evidence suggests that the rate of adoption of basic EHRs has accelerated (6, 7); however, less is knownat a national level about the extent to which physicians areusing these systems to manage their patient panels (8 12).This study uses data from a nationally representative survey of physicians to examine the following questions: What isthe national trend in EHR adoption? How many physi-cians can meet the meaningful use criteria? Which mean-ingful use criteria seem to present the greatest challenges?To what extent are physicians able to use their systems tomanage the health of their patient populations?

See also:

Print

Editorial comment. . . . . . . . . . . . . . . . . . . . . . . . . . 845

Annals of Internal Medicine Original Research

2013 American College of Physicians 791

wnloaded From: http://annals.org/ by a University of Tennessee-Memphis User on 07/30/2013

8/14/2019 0000605-201306040-00003.pdf

2/13

METHODSSample

We obtained our sample from the American Medical Association (AMA) Physician Masterle, which containsbasic demographic and contact information for all physi-

cians in the United States. Physicians were eligible for in-clusion in the sample of primary care physicians (PCPs) if they were board-certied in adolescent medicine, family practice, geriatric medicine, general practice, general pre-ventive medicine, internal medicine, or pediatrics. Physi-cians were eligible for our sample of specialists if they wereboard-certied in a specialty in which they were likely tocare for a given patient over an extended period. Appendix Table 1 (available at www.annals.org) lists the specialtiesincluded in the sample. Military physicians and those working for the U.S. Department of Veterans Affairs wereexcluded.Sample Selection and Size

The National Survey of Physicians is a panel survey, with the rst round of data collection taking place from 19October 2011 to 16 March 2012 and the second plannedfor 2013. Our goal was to have approximately 1000 re-spondents complete both surveys. We determined the sizeof the sample needed for the rst survey by assuming a 10% ineligible rate, a 10% panel attrition rate, and a re-sponse rate of 55% to 60%. Our nal sample of PCPsmeeting our inclusion criteria was 108 515, from which wesampled 2062 (sampling rate of 0.019). Our sample of specialists was 74 050, from which we sampled 1375 (sam-pling rate of 0.019). If 2 physicians were drawn from the

sampling frame with identical addresses (indicating thesame practice), then 1 physician was randomly removed.Instrument Development

The survey was developed by the project team, withreview by a panel of experts that included current (Dr.Farzad Mostashari) and former (Dr. David Blumenthal)

National Coordinators for Health Information Technol-ogy. We used measures for EHR adoption previously de-veloped by the Ofce of the National Coordinator forHealth Information Technology (4), and measures of meaningful use were drawn from policy requirements andother data collection instruments. In addition to the sec-tions focused on health information technology, the survey included measures of care coordination, patient character-istics, use of quality information, and participation in pay-ment incentive programs.Data Collection

Harris Interactive conducted the National Survey of

Physicians by mail between 19 October 2011 and 16March 2012. Physicians were contacted by mail up to 4times and offered a $35 check as an incentive for partici-pation. To meet our target sample goal, the fourth mailing was sent to all physicians who did not respond with anadditional $45 incentive check.Outcome Variables

We examined 2 primary outcome variables: adoptionof a basic EHR and ability to meet meaningful use criteria.Our denition of basic EHR adoption has been described,and the functionalities required to meet this measure areshown in Table 1 (7, 13). It is important to note that the

measure included in our survey of maintaining patient de-mographic characteristics (having a computerized systemfor generating lists of patients by race, ethnicity, or pre-ferred language) is more stringent than survey items usedin other measures of basic EHR adoption where physicians were required only to have an electronic system for main-taining patient demographic characteristics. Because of thisdifference, our estimated proportion of physicians with thisfunction is far lower than that reported by the NationalCenter for Health Statistics (6, 13). Therefore, we have notincluded this measure in our denition of a basic EHR.Our denition of meaningful use contains 3 of the func-

tionalities of a basic EHR and 8 other functions based onCMS criteria for incentive payments (Table 1). The survey items used to create the meaningful use measure are shownin Appendix Table 2 (available at www.annals.org).

Our meaningful use measure also does not include 3of the other CMS criteria necessary for physicians to re-ceive payments. Criteria not included in our survey, andthus not in our measure, are generating and transmitting permissible prescriptions electronically to the pharmacy,maintaining patient allergy lists, and protecting electronichealth information through appropriate technical capabili-ties. In total, our measure of meaningful use contains 11 of the 15 core requirements of stage 1 meaningful use.

Context

The Centers for Medicare & Medicaid Services providesincentive payments to physicians who are able to use anelectronic health record (EHR) to evaluate the care theyprovide their patient populations. Whether most physicianswith EHRs are able to meet criteria for such meaningful

use is unknown.Contribution

In this national survey, approximately 40% of physiciansreported having a basic EHR. Only 1 in 10 physicians re-ported being able to use their systems to meet meaningfuluse criteria, and one half of them believed that it was dif-ficult to do so for at least some criteria (such as generatinglists of patients who were overdue for care).

Caution

Physicians who responded to the survey may differ inunknown ways from those who did not respond.

ImplicationAdoption of an EHR itself does not seem to ensure its useto better manage populations of patients.

The Editors

Original Research Meeting Meaningful Use Criteria and Managing Patient Populations

792 4 June 2013 Annals of Internal Medicine Volume 158 Number 11 www.annals.org


8/14/2019 0000605-201306040-00003.pdf

3/13

Statistical AnalysisTo investigate the possibility of nonresponse bias, we

compared the characteristics of respondents with those inthe AMA Physician Masterle and found slight but statis-tically insignicant differences. We have weighted the data to adjust for these small differences. All analyses were con-

ducted using SAS, version 9.3 (SAS Institute, Cary, NorthCarolina), by researchers at Mathematica Policy Research(Cambridge, Massachusetts) and Massachusetts GeneralHospital (Boston, Massachusetts) using procedures thatadjust for weighted data. The specic procedures used wereproc surveyfreq, proc surveylogistic, and proc sgplot (14).

We used univariate and bivariate analyses to examinecharacteristics of respondents. Next, we compared PCPs with specialists on measures of adoption of individualfunctionalities, adoption of a basic EHR, and ability tomeet our measure of meaningful use. We then applied a logit model to evaluate associations between each of 2outcomesadoption of a basic EHR and ability to meetour proxy measure of meaningful useand characteristicsof physicians (such as age and specialty) and practices (suchas practice size, practice type, and being a member of a larger physician organization).

We next examined barriers to achieving meaningfuluse by determining the functionalities least likely to beadopted by physicians with 8 to 10 of the 11 criteria in-cluded in our meaningful use measure. We then examinedease of use of a computerized system for population man-agement tasks; only respondents with a computerized sys-tem for a given task were included in that analysis. Forexample, in our analysis of the ease of generating lists of

patients who were overdue for certain preventive tests, weincluded only those respondents who reported having a computerized system to do this task. We conducted bivari-ate analyses to examine associations between meeting ourmeaningful use measure and ease of doing populationmanagement tasks. Finally, we examined the relationshipbetween adoption of a basic EHR or ability to meet mean-ingful use criteria and knowledge of the CMS meaningfuluse incentive program and beliefs about the effect of theincreased use of these systems.

Role of the Funding SourceThis work was funded by the Robert Wood Johnson

Foundation and the Commonwealth Fund. Foundationstaff collaborated on the design of the survey instrument,interpretation of the data, and preparation of the manu-script; these staff members are named as authors.

RESULTSCharacteristics of Respondents

After excluding physicians who were deceased, retired,no longer practicing medicine, not practicing in the spe-cialty stated in the AMA Physician Masterle, or otherwiseineligible (193 PCPs and 196 specialists), we received ques-

tionnaires from 1820 respondents, for an overall responserate of 60% (62% among PCPs and 56% among special-ists). As shown in Table 2 , PCPs differed from specialistsin age, sex, race or ethnicity, and annual incomes. Primary care physicians, on average, were younger than specialists were more likely to be women and African American, andreported lower annual incomes. Primary care physicians

were also in practices that were smaller and more likelyto be part of a physician organization or network thanspecialists.

Adoption of EHR FunctionsFigures 1 and 2 show the proportion of physicians

with the individual functionalities that comprise our basicEHR and meaningful use denitions. The most commonly adopted functionalities were viewing laboratory results, or-dering prescriptions electronically, viewing radiology orimaging results, and recording clinical notes. The function-alities that were least likely to be adopted were exchangingpatient clinical summaries and laboratory and diagnostic

Table 1. Health Information Technology Functions Requiredto Meet Basic EHR Adoption and Meaningful Use Criteria

Function Required toMeet Basic EHRAdoption

Measure

Required toMeet Measure ofMeaningful

UseManaging patient clinical information

Maintain patient problem lists X XOrder laboratory testsOrder radiology or imaging testsView laboratory results XView radiology or imaging results XRecord clinical notes XRemind clinicians to provide

guideline-based interventionsX

Record patients smoking status XRecord and chart changes in vital

signsX

Managing medicationsMaintain a patients active medication

list

X X

Provide warnings of drug interactionsor contraindications

X

Order prescriptions electronically X X

Data exchangeExchange patient clinical summaries X*Exchange laboratory and diagnostic

testsX*

Patient engagement Provide patients with electronic copies

of their health informationX

Provide patients with after-visitsummaries

X

Generating quality metrics X

EHR electronic health record.* Physicians reporting exchange of clinical summaries or laboratory and diagnostictests were considered to have met the meaningful use data exchange criteria.

Original ResearchMeeting Meaningful Use Criteria and Managing Patient Populations

www.annals.org 4 June 2013 Annals of Internal Medicine Volume 158 Number 11 793


8/14/2019 0000605-201306040-00003.pdf

4/13

test results with outside entities, generating quality metrics,and providing patients with after-visit summaries and cop-ies of their health information. Appendix Table 3 (avail-able at www.annals.org) shows completed survey results forall health information technology functions.

Approximately 4 in 10 PCPs (44.9%) and specialists(41.2%) met the criteria for having a basic EHR. In mul-tivariate analyses, adoption of a basic EHR was statistically associated with group size and network membership. As

shown in Appendix Table 4 (available at www.annals.org),physicians in larger groups and members of a larger physi-cian organization were all more likely than their counter-parts to have a basic EHR. Approximately two thirds of physicians (65%) reported adopting an EHR before 2009.This did not vary signicantly by specialty; however, phy-sicians in larger practices were more likely than those insmaller practices to report adopting before 2009 (P 0.001).

Table 2. Physician Respondent Characteristics*

Characteristic Total ( n 1820) PCP ( n 1164) Specialist ( n 656) P Value

Age 0.0102539 y 10.2 9.7 10.94049 y 32.1 35.2 27.55059 y 34.0 33.4 34.9

60 y 21.6 19.9 24.1Declined to answer 2.1 1.8 2.5

Sex 0.001Male 64.6 56.4 76.9Female 34.8 43.2 22.4Declined to answer 0.5 0.3 0.7

Race or ethnicity 0.041White, non-Hispanic 64.2 63.3 65.6Black, non-Hispanic 3.1 4.3 1.4Asian, non-Hispanic 18.2 18.2 18.1Other, non-Hispanic 3.6 3.2 4.1Hispanic 4.8 5.2 4.3Declined to answer 6.1 5.7 6.5

Annual income 0.001$100 000 8.1 11.2 3.4

$100 000$199 999 36.1 48.4 17.8$200 000$299 999 24.1 23.6 24.8$300 000$399 999 11.8 8.2 17.2

$400 000 13.7 4.3 27.7Declined to answer 6.0 4.0 8.8

Practice site 0.001Private solo or physician group practice 70.1 67.4 74.1Community clinic or health center 2.9 4.2 1.1Ambulatory center 8.7 9.4 7.5On site at hospital or medical center 11.7 11.1 12.5Walk-in care center 0.3 0.4 0.2Other 3.3 4.0 1.9Declined to answer 3.1 3.3 2.7

Practice size 0.033Solo practice 29.1 29.8 27.824 physicians 38.1 38.9 36.8510 physicians 13.4 13.3 13.7

11 physicians 14.2 12.2 17.6Declined to answer 4.9 5.5 3.8

Provider organizationLarge, multispecialty group practice 27.7 29.3 25.4 0.167Integrated delivery system 29.6 30.9 27.7 0.133Independent practice association 14.1 15.4 12.1 0.008Physician hospital organization 17.8 18.4 17.1 0.011Other network of physicians practices 10.9 11.4 10.3 0.158Not part of any physician network or integrated delivery system 38.6 35.0 43.9 0.001

PCP primary care physician.* Values reported are percentages. Percentages may not sum to 100 due to rounding. P values are for comparisons of PCPs and specialists. Percentages will not sum to 100 because physicians were asked to respond yes or no to each practice organization type.




8/14/2019 0000605-201306040-00003.pdf

5/13

8/14/2019 0000605-201306040-00003.pdf

6/13

To identify the most challenging functions for physi-cians who could not meet our meaningful use criteria in2012, we examined those who were close to meeting thestandard: physicians with 8 and 10 of the required func-tions. Among this group, approximately 4 in 10 physicianscould not electronically exchange data with physicians out-side of their practice (41.6%), could not generate quality metrics (40.8%), and could not provide patients with anafter-visit summary (36.2%).

Participation in the Meaningful Use ProgramMore than one half of respondents either were not

planning to participate in the meaningful use incentiveprogram (17.7%) or did not know whether they wouldparticipate (36.1%), whereas 25.1% had registered for pay-ments. This does not vary signicantly by specialty or timesince rst adopting a system; however, physicians with a basic EHR and those meeting meaningful use criteria werestatistically signicantly more likely to report either regis-

Figure 3. Number of meaningful use functions reported by PCPs and specialists.

P C P s ,

%

Meaningful Use Functions, n

3.3

0 1 2 3 4 5 6 7 8 9 10 11 12

5

10

15

2.9

3.6 3.7

7.1

12.7

15.5

12.3

11.2

5.9

7.3

14.6

S p e c

i a l i s

t s , %

Meaningful Use Functions, n

3.8

0 1 2 3 4 5 6 7 8 9 10 11 12

5

10

20

15

3.5 3.3

4.6

9.9

11.4

15.8

9.3

7.6

6.2

12.412

PCP primary care physician.




8/14/2019 0000605-201306040-00003.pdf

7/13

tering or planning to register for payments (P 0.001). Approximately 12.2% of all physicians reported attesting to meaningful use at the time the survey was completed.Using the EHR for Population Management

Table 3 displays the availability and ease of use of computerized systems for patient population management.

Less than one half of physicians reported having a comput-erized system for generating lists of patients by diagnosis. Approximately one third or fewer physicians had an elec-tronic system for tracking referral completion, generating reports on quality of care, sending patient reminders forpreventive or follow-up care, generating lists of patients who have missed appointments or were overdue for care,generating lists of patients by laboratory result, and provid-ing patients with after-visit summaries.

Physicians varied in their perceptions of how easy these computerized systems for patient population man-agement were to use. Approximately one half of physicians

with the respective computerized systems reported thatthey could not or that it was very or somewhat difcult togenerate lists of patients by laboratory result, lists of pa-tients who were overdue for care, and reports on quality of care and track referrals. Physicians with EHRs who metour meaningful use standard were statistically signicantly more likely than those not meeting the standard to rate allpanel management tasks as easy ( Appendix Table 6, avail-able at www.annals.org).Beliefs About the Effect of Increased Use of EHRs

Physicians varied in their level of familiarity withmeaningful use criteria and beliefs about the effects of

greater use of EHRs on the larger health care system (Table4). Physicians with at least a basic EHR were more likely than those without such systems to report being veryfamiliar with the criteria. Primary care physicians weremore likely than specialists to report that the increased useof EHRs would have a positive effect on the quality andefciency of health care in the United States. As shown inTable 4 , physicians with at least a basic EHR were signif-icantly more likely than those without to hold positiveopinions about the effect of increased use of EHRs.

DISCUSSIONIn this article, we report on ndings from a new na-

tional survey of practicing physicians. Findings suggest thatthe rate of adoption of a basic EHR is continuing to in-crease. Using a measure that is very similar to the one usedby the National Ambulatory Medical Care Survey, we

found that EHR adoption among physicians increasedfrom nearly 34% in early 2011 to 43.5% by March 2012,indicating substantial growth. However, a signicantly smaller proportion of physicians seems to be able to meetstage 1 meaningful use criteria (6). Computerized systemsfor patient panel management and quality reporting do notseem widespread, and where they are implemented physi-cians reported that they are not always easy to use. Physi-cians with EHRs that meet our meaningful use standardreport that population management tasks are easy to do atmuch higher rates than those with less comprehensive sys-tems; however, substantial proportions still reported thatthese tasks are difcult.

Our ndings on the proportion of U.S. physicians with a basic EHR indicate a substantially slower rate ofadoption than a recent report nding that 54% of physi-cians had an EHR in 2011 (15). The disparate ndings arelikely related to differences in how EHR adoption is de-ned. The report uses 1 general question on the type of record used in the practice to dene EHR adoption. Earlierresearch has suggested that this approach to measuring adoption can overestimate the proportion of physicians with functional EHRs (16). Our signicantly lower rate of adoption is more in line with that reported in an article by Hsiao and colleagues (7), in which a functionality-baseddenition of EHR adoption was used that is very similar toours.

Results on the perceived ease of use of EHR functionsreect widespread criticisms of the usability of these tools(1719). The Ofce of the National Coordinator forHealth Information Technology has begun to address theseconcerns with the development of guidelines to assess us-ability as part of the EHR certication program; however,

Table 3. Ease of Panel Management Tasks Among Physicians With Computerized Systems

Task Practice Has aComputerized Systemfor the Task, n (%)

Ease-of-Use Response, %

Easy Somewhat Difficult

Difficult or Cannot Generate

Do Not Know or Not Applicable

Generate list of patients by diagnosis 808 (44.4) 60.9 25.8 11.8 1.4Generate list of patients by laboratory result 571 (31.4) 43.8 29.7 24.7 1.7Generate list of patients who were overdue for tests or preventive

care or have missed appointments621 (34.1) 50.9 27.1 20.7 1.2

Generate reports on quality-of-care measures 571 (31.4) 51.4 29.1 18.1 1.4List patients race, ethnicity, or preferred language 617 (33.9) 54.6 24.9 15.6 2.1Provide patients with after-visit summaries 606 (33.3) 75.7 13.8 9.0 1.5Track referral completion 520 (28.6) 48.2 29.6 20.5 1.8Send patients reminders for preventive or follow-up care 576 (31.6) 59.6 22.7 16.5 1.2




8/14/2019 0000605-201306040-00003.pdf

8/13

it seems that signicant progress needs to be made beforesuch systems are believed to be usable by most physicians.

Our study had several limitations. First, like all sur-veys, ours was subject to potential response bias. Physicians who responded to our survey may have had a greater in-

terest in the subject or differed in other important waysfrom those who did not respond. To the extent that theseare systematic differences between the 2 groups rather thanrandom nonresponse, bias could exist. We have weightedthe data to account for potential nonresponse bias; how-ever, these adjustments are imperfect and cannot accountfor the possibility that physicians who did not respond tothe survey were less enthusiastic about health informationtechnology than those who did respond. Second, although we adhered to the best practices of survey research to en-sure that we reached the intended respondents, we cannotverify that the questionnaires were completed by the phy-

sicians themselves. Third, we cannot conrm the accuracy of respondents reports on the availability or ease of use of EHR functions. Fourth, our proxy measure of meaningfuluse does not include all of the stage 1 core and menucriteria. Although our measure is more comprehensive thanothers in the literature, our approximation may overesti-mate the proportion of physicians who were able to meetthe CMS meaningful use criteria at the time the survey wascompleted (7, 20). In addition, we cannot account for thesmall difference in the proportion of physicians who at-tested to meaningful use and those who met our meaning-ful use criteria; however, the difference falls within themargin of error for a sample of this size.

Finally, estimates of the ease of use of tools for patientpopulation management among physicians meeting ourcriteria for meaningful use are based on a few respondentsand are subjective. By virtue of being able to meet mean-ingful use criteria at this stage, these physicians may mark-

edly differ from respondents who did not meet the proxy measure. Any extrapolation of benets to a larger popula-tion of physicians should be done with caution.

Results from this research have implications for policy and future research. Payments for meaningful use, whichbegan in 2011, have been made to more than 145 000health care providers (approximately 20% of all eligibleproviders) through Medicare and Medicaid and, by Sep-tember 2012, totaled more than $3.9 billion (21, 22).Reaching physicians who have not yet met the standardmay be more difcult because previous work has shownthat early adopters of health information technology differ

from later adopters in personal and practice characteristics(6, 13, 23). Given that many physicians have at least a fewfunctions in place, a promising strategy may be to focus onthe distribution of adoption of individual meaningful usefunctions.

Our ndings around the perceived usability of thesecomputerized systems for patient panel management sug-gest that this is an important area for future research. Poorimplementation, lack of training, the need to upgrade sys-tems, and complicated procedures (for example, navigating through several screens) can all affect physicians percep-tions and use of such systems. As the systems become more widespread, research on usability and optimal implemen-

Table 4. Attitudes Toward Meaningful Use and EHRs*

Response Total(n 1820)

PCPs(n 1164)

Specialists(n 656)

P Value Basic EHR(n 792)

No Basic EHR(n 1028)

P Value

Familiarity with meaningful use criteria 0.98 0.001Very familiar 21.7 21.8 21.6 30.3 15.2Somewhat familiar 49.2 49.3 49.1 50.4 48.4Not too familiar 20.0 20.2 19.7 12.4 26.0Not familiar at all 6.7 6.4 7.2 4.3 8.7Do not know 2.2 2.1 2.3 2.6 1.7

Effect of increased use of EHRs on the followingQuality of care delivered in the United States 0.001 0.001

Positive effect 57.1 61.8 50.1 67.5 49.1No effect 27.9 25.7 31.2 20.8 33.4Negative effect 12.9 9.8 17.5 9.3 15.6Do not know 2.0 2.6 1.2 2.2 1.8

Cost of care in the United States 0.070 0.001Positive effect 33.5 35.6 30.3 42.1 26.8No effect 26.5 26.1 27.1 25.6 27.2Negative effect 38.0 36.1 40.9 30.1 44.1Do not know 2.0 2.2 1.7 2.1 1.9

Efficiency of care in the United States 0.001 0.001Positive effect 55.7 59.9 49.3 49.4 63.8

No effect 22.9 21.3 25.5 26.9 17.8Negative effect 19.3 16.2 23.8 21.8 16.0Do not know 2.1 2.6 1.4 1.9 2.1

EHR electronic health record; PCP primary care physician.* Values reported are percentages. Percentages may not sum to 100 due to rounding. P values are for comparisons of PCPs and specialists. P values are for comparisons of physicians with and without basic EHRs.




8/14/2019 0000605-201306040-00003.pdf

9/13

tation could provide important insights into methods forhelping physicians move beyond using EHRs as replace-ments for paper records to accessing the full potential of these tools.

The pace of adoption of basic EHRs seems to be in-creasing; however, ndings around availability and per-

ceived ease of use of systems that can help to manage pa-tient populations should be of concern to policymakers.Using EHRs as simple replacements for the paper record will not result in the gains in quality and efciency or thereductions in cost that EHRs have the potential to achieve.On the other hand, when physicians and others can useand take advantage of the full scope of the EHRs func-tionalities, they may be more likely to improve the quality,efciency, and patient-centeredness of the care they deliver.

From Mathematica Policy Research, Cambridge, Massachusetts; Com-monwealth Fund, New York, New York; Robert Wood Johnson Foun-dation, Princeton, New Jersey; and Mongan Institute for Health Policy,

Massachusetts General Hospital, Boston, Massachusetts.

Note: Drs. DesRoches and Donelan had full access to all of the data inthe study and take responsibility for the integrity of the data and accu-racy of the analysis.

Acknowledgment: The authors thank John McCauley (Mathematica Policy Research) and Samantha Stalley (Mathematica Policy Research)for their contributions to the analysis and reporting of the data.

Grant Support: By the Robert Wood Johnson Foundation (grant68812) and the Commonwealth Fund (grant 20110087).

Potential Conflicts of Interest: Disclosures can be viewed at www

.acponline.org/authors/icmje/ConictOfInterestForms.do?msNumM12-2881.

Reproducible Research Statement: Study protocol: Available from Dr.DesRoches (e-mail, [email protected]). Statistical code and data set: Not available.

Requests for Single Reprints: Catherine M. DesRoches, DrPH, Math-ematica Policy Research, 955 Massachusetts Avenue, Suite 800, Cam-bridge, MA 02139; e-mail, [email protected].

Current author addresses and author contributions are available at www .annals.org.

References1. Blumenthal D. Implementation of the federal health information technology initiative. N Engl J Med. 2011;365:2426-31. [PMID: 22187990]2. Centers for Medicare & Medicaid Services. Medicare & Medicaid EHR Incentive Program: Meaningful Use Stage 1 Requirements Overview 2010. Ac-cessed at www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/MU_Stage1_ReqOverview.pdf on 2 April 2013.3. Blumenthal D, Tavenner M. The meaningful use regulation for electronichealth records. N Engl J Med. 2010;363:501-4. [PMID: 20647183]

4. DesRoches CM, Jha AK, Painter M, eds. Health Information Technology inthe United States: Moving Toward Meaningful Use, 2010. Princeton, NJ: Rob-ert Wood Johnson Foundation; 2010. Accessed at www.rwjf.org/content/dam/farm/reports/reports/2010/rwjf68967 on 2 April 2013.5. Ofce of the National Coordinator for Health Information Technology.Meaningful Use Stage 2. Accessed at www.healthit.gov/policy-researchers-implementers/meaningful-use-stage-2 on 23 January 2013.6. Decker SL, Jamoom EW, Sisk JE. Physicians in nonprimary care and small

practices and those age 55 and older lag in adopting electronic health recordsystems. Health Aff (Millwood). 2012;31:1108-14. [PMID: 22535502]7. Hsiao CJ, Decker SL, Hing E, Sisk JE. Most physicians were eligible forfederal incentives in 2011, but few had EHR systems that met meaningful-usecriteria. Health Aff (Millwood). 2012;31:1100-7. [PMID: 22535501]8. Dinakarpandian D, Lee Y, Dinakar C. Applications of medical informatics inallergy/immunology. Ann Allergy Asthma Immunol. 2007;99:2-9. [PMID:17650823]9. Friedberg MW, Coltin KL, Safran DG, Dresser M, Zaslavsky AM, Sch-neider EC. Associations between structural capabilities of primary care practicesand performance on selected quality measures. Ann Intern Med. 2009;151:456-63. [PMID: 19805769]10. Gill JM, Chen YX, Grimes A, Klinkman MS. Using electronic healthrecord-based tools to screen for bipolar disorder in primary care patients withdepression. J Am Board Fam Med. 2012;25:283-90. [PMID: 22570391]

11. Menachemi N, Lee SC, Shepherd JE, Brooks RG. Proliferation of electronichealth records among obstetrician-gynecologists. Qual Manag Health Care.2006;15:150-6. [PMID: 16849986]12. Wright A, Pang J, Feblowitz JC, Maloney FL, Wilcox AR, McLoughlin KS,et al. Improving completeness of electronic problem lists through clinical decisionsupport: a randomized, controlled trial. J Am Med Inform Assoc. 2012;19:555-61. [PMID: 22215056]13. DesRoches CM, Campbell EG, Rao SR, Donelan K, Ferris TG, Jha A,et al. Electronic health records in ambulatory carea national survey of physi-cians. N Engl J Med. 2008;359:50-60. [PMID: 18565855]14. SAS Institute. Base SAS 9.3, Procedures Guide. Cary, NC: SAS Institute;2012.15. Jamoom E, Beatty P, Bercovitz A, Woodwell D, Palso K, Rechtsteiner E.Physician Adoption of Electronic Health Record Systems: United States, 2011.NCHS data brief, no. 98. Hyattsville, MD: National Center for Health Statistics;2012. Accessed at www.cdc.gov/nchs/data/databriefs/db98.htm on 2 April 2013.16. Jha AK, Ferris TG, Donelan K, DesRoches C, Shields A, Rosenbaum S,et al. How common are electronic health records in the United States? A sum-mary of the evidence. Health Aff (Millwood). 2006;25:w496-507. [PMID:17035341]17. Edwards PJ, Moloney KP, Jacko JA, Sainfort F. Evaluating usability of a commercial electronic health record: a case study. Int J Hum Comput Stud.2008;66:718-28.18. Guerra A. Healthcare providers voice gripes at EHR usability hearing.InformationWeek; 2011. Accessed at www.informationweek.com/healthcare/electronic-medical-records/healthcare-providers-voice-gripes-at-ehr/229402180on 9 November 2012.19. Fiegl C. EHR certication lacking usability factor, doctors say [news release].Chicago: American Med Assoc; 4 June 2012. Accessed at www.amednews.com/article/20120604/government/306049953/4 on 2 April 2013.20. Hogan SO, Kissam SM. Measuring meaningful use. Health Aff (Millwood).2010;29:601-6. [PMID: 20368588]21. Centers for Medicare & Medicaid Services. Combined Medicare and Med-icaid Payments by State. Accessed at www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Downloads/September_PaymentsbyStatesbyProgrambyProvider.pdf on 7 November 2012.22. Bruen BK, Ku L, Burke MF, Buntin MB. More than four in ve ofce-based physicians could qualify for federal electronic health record incentivesHealth Aff (Millwood). 2011;30:472-80. [PMID: 21383366]23. Burt CW, Sisk JE. Which physicians and practices are using electronic med-ical records? Health Aff (Millwood). 2005;24:1334-43. [PMID: 16162581]




8/14/2019 0000605-201306040-00003.pdf

10/13

Current Author Addresses: Dr. DesRoches: Mathematica Policy Re-search, 955 Massachusetts Avenue, Suite 800, Cambridge, MA 02139.Dr. Audet: Commonwealth Fund, 1 East 75th Street, New York,NY 10021.Dr. Painter: Robert Wood Johnson Foundation, PO Box 2316, Route 1and College Road East, Princeton, NJ 08543.Ms. Donelan: Mongan Institute for Health Policy, Massachusetts Gen-

eral Hospital, 50 Staniford Street, Suite 900, Boston, MA 02114.

Author Contributions: Conception and design: C.M. DesRoches, A.M. Audet, M. Painter, K. Donelan. Analysis and interpretation of data: C.M. DesRoches, A.M. Audet, M.Painter, K. Donelan.Drafting of the article: C.M. DesRoches, A.M. Audet, M. Painter,K. Donelan.Critical revision of the article for important intellectual content: C.M.

DesRoches, A.M. Audet, K. Donelan.Final approval of the article: C.M. DesRoches, A.M. Audet, M. Painter,K. Donelan.Provision of study materials or patients: K. Donelan.Statistical expertise: C.M. DesRoches, K. Donelan.Obtaining of funding: C.M. DesRoches, M. Painter, K. Donelan. Administrative, technical, or logistic support: C.M. DesRoches,K. Donelan.

Appendix Table 1. Specialists in the Physician Sample

AllergyAllergy and immunologyHematologyHepatologyHematology and oncology

Interventional cardiologyInfectious diseasesImmunologyInternal medicine: Cardiac electrophysiologyInternal medicine: GeriatricsInternal medicine: Preventive medicineInternal medicine: Sports medicineInternal medicine: NeurologyInternal medicine: PsychiatryInternal medicine: Physical medicine and rehabilitationNeurologyNephrologyPediatric allergyPediatric cardiologyPediatric endocrinologyPediatric infectious diseasesPediatric nephrologyPediatric rheumatologyPulmonary diseaseReproductive endocrinologyRheumatologyVascular medicineCardiovascular diseaseChild neurologyDermatologyDevelopmental: Behavioral pediatricsDiabetesEndocrinologyGastroenterologyPediatric pulmonologyPediatric gastroenterologyPediatric hematology and oncology

Annals of Internal Medicine

www.annals.org 4 June 2013 Annals of Internal Medicine Volume 158 Number 11 W-323


8/14/2019 0000605-201306040-00003.pdf

11/13

Appendix Table 2. Survey Questions Used for Meaningful Use Functions

Meaningful Use Criterion Survey Questions

Use CPOE Does your practice site have a computerized system for ordering prescriptions electronically?Implement drugdrug and drugallergy

interaction checksDoes your practice site have a computerized system for providing warnings of drug interactions or

contraindications when the prescription is written?Record demographic characteristics (sex, race,

ethnicity, date of birth, and preferredlanguage)

No measure

Maintain up-to-date problem list of currentand active diagnoses

Does your practice site have a computerized system for maintaining patient problem lists?

Maintain active medication list Does your practice site have a computerized system for maintaining a patients active medication list,including medications prescribed by other clinicians?

Record and chart changes in vital signs(height, weight, blood pressure, body massindex, and growth charts for children)

Does your practice site have a computerized system for recording and charting changes in vital signs(e.g., blood pressure) and body mass index?

Record smoking status Does your practice site have a computerized system for recording patients smoking status?Implement 1 CDS rule relevant to specialty or

high clinical priority and the ability to trackcompliance to that rule

Does your practice site have a computerized system for reminding clinicians to provide guideline-basedinterventions?

Report ambulatory clinical measure to CMSor the state

With the patient medical records system you currently have, how easy would it be to generate reports onquality-of-care measures, such as hemoglobin A 1c , blood pressure control, or smoking? Is this processcomputerized?

Provide clinical summaries for patients for each office visit

With the patient medical records system you currently have, how easy would it be to provide patientswith after-visit summaries? Is this process computerized?

Capability to exchange key clinicalinformation among providers of care andpatient-authorized entities electronically

Using the information technology currently available in your practice, can you electronically exchangepatient clinical summaries or laboratory and diagnostic tests with physicians outside your practice?*

Generate and transmit permissibleprescriptions electronically

No measure

Maintain active medication al lergy l ist No measureOn request, provide patients with electronic

copies of their health informationDoes your practice have a computerized system that allows patients to download copies of their health

information?Protect electronic health information created

or maintained by the certified EHRtechnology through implementation ofappropriate technical capabilities

No measure

CDS clinical decision support; CMS Centers for Medicare & Medicaid Services; CPOE computerized physician order entry; EHR electronic health record.* Physicians who indicated at least 1 of these 3 capabilities were considered to have met the meaningful use criterion.

W-324 4 June 2013 Annals of Internal Medicine Volume 158 Number 11 www.annals.org


8/14/2019 0000605-201306040-00003.pdf

12/13

Appendix Table 3. Adoption of Health Information Technology Functions

Function Physicians Who Adopted the Functions, % P Value

Total ( n 1820) PCP ( n 1164) Specialist ( n 656)

Managing patient clinical informationGenerating lists of patients by demographic characteristics* 33.9 36.5 29.1 0.043Maintaining patient problem lists 64.3 65.4 62.5 0.47Ordering laboratory tests 61.0 64.1 55.6 0.005Ordering radiology or imaging tests 52.7 54.3 49.9 0.29Viewing laboratory results 80.2 83.2 75.0 0.001Viewing radiology or imaging results 67.4 69.3 63.9 0.121Recording clinical notes 67.0 67.1 66.8 0.83Reminding clinicians to provide guideline-based interventions 42.7 44.5 39.5 0.20Recording patients smoking status 61.5 62.9 59.0 0.22Recording and charting changes in vital signs 64.9 66.3 62.4 0.167

Managing medicationsMaintaining a patients active medication list 64.1 64.9 62.7 0.75Provide warnings of drug interactions or contraindications 61.6 63.6 58.1 0.033Ordering prescriptions electronically 74.2 71.8 78.5 0.021

Data exchange

Exchange patient clinical summaries 34.5 33.9 35.5 0.32Exchange laboratory and diagnostic tests 37.1 37.6 36.1 0.52

Patient engagement Provide patients with after-visit summaries 33.0 35.9 28.6 0.003Provide patients with electronic copies of their health information 13.5 14.9 10.9 0.018

Quality measurement Generating quality metrics 31.3 35.1 24.7 0.001

Summary measuresBasic EHR 43.5 44.9 41.2 0.138All meaningful use functions 9.8 11.2 7.6 0.015

EHR electronic health record; PCP primary care physician.* This measure is not included in the denition of a basic EHR or as part of the meaningful use criteria variable because it is much more stringent than measures used oother surveys, such as the National Ambulatory Medical Care Survey, which require physicians only to have a computerized system for maintaining patient demographcharacteristics.

Appendix Table 4. Logistic Regression Results About theLikelihood of a Physician Having a Basic Electronic HealthRecord

Characteristic Odds Ratio (95% CI) P Value

Age 1.01 (0.961.12) 0.114

Physician typeSpecialist ReferencePCP 1.18 (0.961.47) 0.111

Practice typePrivate solo or group ReferenceHospital 1.28 (0.981.67) 0.067Community clinic or community

health center 0.63 (0.341.18) 0.147

Other 1.26 (0.772.06) 0.35

Practice sizeSmall ReferenceMedium 1.92 (1.512.45) 0.001Large 4.17 (3.125.57) 0.001

Practice is part of a larger organization 2.52 (2.003.17) 0.001


Appendix Table 5. Logistic Regression Results About theLikelihood of a Physician Having a System MeetingMeaningful Use Criteria

Characteristic Odds Ratio (95% CI) P Value

Age 1.01 (0.981.12) 0.89

Physician typeSpecialist ReferencePCP 1.57 (1.112.23) 0.011

Practice typePrivate solo or group ReferenceHospital 0.73 (0.491.09) 0.127Community clinic or community

health center 0.26 (0.061.05) 0.057

Other 0.91 (0.061.05) 0.81

Practice sizeSmall ReferenceMedium 1.76 (1.152.72) 0.009Large 2.68 (1.664.32) 0.001

Practice is part of a larger organization 1.87 (1.232.79) 0.002


www.annals.org 4 June 2013 Annals of Internal Medicine Volume 158 Number 11 W-325


8/14/2019 0000605-201306040-00003.pdf

13/13

Appendix Table 6. Physician Responses About Ease of Use, by Meaningful Use Status*

With the Patient Medical Records System You Currently Have, How Easy Would It Be for You(or Staff in Your Practice) to Do the Following?

Physician Has a SystemThat Meets Meaningful Use

Criteria

P Value

Yes No

Generate list of patients by diagnosis n 166 n 642 0.001Easy 76.3 56.9Somewhat difficult 16.2 28.3Difficult or cannot generate 4.8 13.6Do not know or not applicable 2.5 1.1

Generate list of patients by laboratory result n 155 n 416 0.001Easy 60.3 37.6Somewhat difficult 26.4 30.9Difficult or cannot generate 11.8 29.4Do not know or not applicable 1.4 1.9

Generate list of patients who were overdue for tests or preventive care or have missed appointments n 164 n 457 0.001Easy 62.9 46.5Somewhat difficult 27.5 27.0Difficult or cannot generate 8.3 25.2

Do not know or not applicable 1.3 1.2

Generate reports on quality-of-care measures n 167 n 404 0.001Easy 70.5 43.5Somewhat difficult 23.9 31.2Difficult or cannot generate 4.4 23.8Do not know or not applicable 1.3 1.5

List patients race, ethnicity, or preferred language n 167 n 450 0.001Easy 76.6 54.5Somewhat difficult 13.0 24.9Difficult or cannot generate 7.9 18.5Do not know or not applicable 2.5 2.1

Provide patients with after-visit summaries n 167 n 439 0.001Easy 87.7 71.1Somewhat difficult 6.6 16.5Difficult or cannot generate 3.8 11.0Do not know or not applicable 1.9 1.4

Track referral completion n 151 n 369 0.004Easy 56.7 44.7Somewhat difficult 31.3 28.9Difficult or cannot generate 9.8 24.8Do not know or not applicable 2.1 1.6

Send patients reminders for preventive or follow-up care n 152 n 424 0.005Easy 71.4 55.5Somewhat difficult 18.8 24.0Difficult or cannot generate 8.4 19.4Do not know or not applicable 1.4 1.1

* Values reported are percentages. Percentages may not sum to 100 due to rounding. P values are for comparisons between physicians meeting our meaningful use proxy criteria and those who did not meet the criteria.

W 326 4 June 2013 Annals of Internal MedicineVolume 158 Number 11 www annals org

Documents

0000605-201306040-00003.pdf