Editorial Comment

Two Years of Appropriateness Criteria for Echocardiography:What Have We Learned and What Else Do We Need to Do?

Nicole M. Martin, MD, and Michael H. Picard, MD, FASE, Boston, Massachusetts

As noninvasive cardiovascular imaging has become more sophisti-cated and accessible, its use has increased steadily. In the UnitedStates, according to Medicare data, transthoracic echocardiography(TTE) constitutes a significant proportion of cardiovascular imaging.1

This modality is used in a variety of clinical settings and providesa great deal of information at virtually no risk to patients. BecauseTTE is seen as benign and is widely available, the potential for overuseexists. By developing echocardiographic appropriateness criteria(AC), the American College of Cardiology Foundation and the Amer-ican Society of Echocardiography have provided a framework ofbroad indications that encompasses many clinical scenarios. The goalsof the AC are to provide a rational approach to the use of cardiovas-cular imaging and to improve patient care and health outcomes ina cost-effective manner.2 Ideally, AC should help providers moderatetheir own practice patterns without stifling clinical judgment, reducingif not supplanting the need for external regulation by insurers andother third parties. These are particularly important goals given thecurrent dialogue on health care reform, as it appears that there aremany parties that wish to control how imaging is utilized in an effortto reduce costs.

AC are different from clinical guidelines. Clinical guidelines de-scribe a comprehensive list of the conditions for which a test is indi-cated. AC include more selective indications than guidelines,typically those that are most commonly encountered (because thesehave the largest potential to generate inappropriate studies). Many cli-nicians might consider a test to be appropriate if it provides importantnew information and inappropriate if it does not provide any new in-formation. However, in the AC, an appropriate test is defined as ‘‘onein which the expected incremental information, combined with clin-ical judgment, exceeds the expected negative consequences by a suf-ficiently wide margin for a specific indication that the procedure isgenerally considered acceptable.’’2 This means that the AC may cate-gorize some tests as ‘‘appropriate’’ even though the results simply con-firm what is already known, while other tests may be categorized as‘‘inappropriate’’ even though they do in fact provide new informationthat might be valuable for patient care. Also, there will be scenarios inwhich it is uncertain or unclear if a test is appropriate. Because the ACare not all inclusive, some tests will not fit into one of the coveredscenarios and thus will be ‘‘unclassifiable.’’

The AC for TTE and transesophageal echocardiography (TEE) werepublished in July 2007,2 and those for stress echocardiography ap-

From the Division of Cardiology (M.H.P.) and the Department of Medicine (N.M.M.,

M.H.P., Massachusetts General Hospital and Harvard Medical School, Boston,

Massachusetts.

Editorial Comments published in the Journal of the American Society of Echo-

cardiography (JASE) reflect the opinions of their author(s), and do not neces-

sarily represent the views of JASE, its editors, or the American Society of

Echocardiography.

Reprint requests: Michael H. Picard, MD, Massachusetts General Hospital, Car-

diac Ultrasound Laboratory, Yawkey 5E, 55 Fruit Street, Boston, MA 02114.

(E-mail: mhpicard@partners.org).

0894-7317/$36.00

doi:10.1016/j.echo.2009.05.024

800

peared 8 months later.3 Relatively quickly after the publication of theseAC, single-center studies began to address a variety of issues regardingTTE, stress echocardiography, and TEE in outpatient and inpatient set-tings, mostly from academic medical centers. There are many lessonsto be learned from these reports, and consistent messages.

PREVIOUS EXPERIENCES WITH THE APPROPRIATENESS

CRITERIA

To be useful in practice, AC must be relatively easy to apply. Althoughtwo previous studies showed that a large majority of transthoracicechocardiographic studies were classified as appropriate,4,5 a thirdstudy by Kirkpatrick et al6 found more than one third of such studiesto be unclassifiable by the AC clinical scenarios. This discrepancyhighlights the variable uses of TTE by different practitioners and thediverse types of patients encountered at different institutions. Cer-tainly the AC contain several gaps, and the intent of the writing groupis for the AC documents to be refined over time. For instance, the ACdo not address the routine use of TTE before, during, and after spe-cialized procedures and therapies such as cardiac transplantation,mechanical support in advanced heart failure, and cardiac resynchro-nization therapy or in the evaluation of bradycardia.4-6

Despite the differences among prior studies, it is reassuring that therates of inappropriate ordering of TTE are all quite similar, clusteringaround 10%. The rate of inappropriate stress echocardiography wasreported to be 18% at one academic medical center.7 We do notknow, however, how accurately all these data reflect the general prac-tice of echocardiography. Because the studies published to date havecome from large academic cardiology practices, whereas the majorityof transthoracic echocardiographic studies are ordered by general-ists,1 it will be illuminating in the future to study the AC formally insettings other than academic medical centers.

The reports so far have shown that TTE is much more likely to beappropriate when performed on inpatients.4-6 Because many of theappropriate indications hinge on there being a recent change in clin-ical status, this is not a surprising finding. Patient age also has a signif-icant impact on the appropriateness of TTE; younger patients aremore likely to undergo inappropriate TTE4 and less likely to haveclinically significant findings.6

Differences in reimbursement may influence test-ordering prac-tices. A goal of the AC is to remove such issues from consideration.Given the wide variety of practice settings in the American healthcare system, TTE ordering might be affected in any number of waysthat have not been elucidated. Thus, again, it is critical to examinehow the AC perform in as broad an environment as possible.

Similarly, the influence of provider training level and expertise onthe appropriateness of TTE is not yet fully understood. We have dem-onstrated that internal medicine house officers follow AC quite wellbut that cardiology consultation can have a positive impact on the ap-propriateness of TTE.5 Ward et al4 included a mix of specialists andgeneralists in their work and showed that noncardiac specialistswere significantly more likely to order inappropriate TTE than cardiacspecialists.

Journal of the American Society of EchocardiographyVolume 22 Number 7

Martin and Picard 801

Most recently, adherence to the AC for TEE has been studied.8,9

Compared with prior observations of TTE ordering, a larger propor-tion of the transesophageal echocardiographic studies ordered weredeemed appropriate, and thus there were fewer classified asinappropriate or uncertain. The studies noted that even transesopha-geal echocardiographic studies not classifiable into one of the scenar-ios still affected management most of the time. Notably, the majorityof the cases studied were inpatients, and most transesophageal echo-cardiographic studies were ordered by cardiologists. Still, up to 9% didnot fit one of the scenarios, again pointing out the need for additionalindications in the AC.

In summary, to date, single-center studies have shown that the ACfor echocardiography are practical and easy to apply. Not all testsordered can be classified into the current scenarios, but among thosethat are classifiable, about 10% have been deemed inappropriate.Higher rates of appropriateness have been found in older patientsand among inpatients.

A NEW CONTRIBUTION

This issue of JASE contains the latest addition to the growing body ofliterature on echocardiography appropriateness. Willens et al10 com-pared TTE ordering at the echocardiography laboratories of theUniversity of Miami’s academic medical group practice and at a USDepartment of Veterans Affairs (VA) medical center, including both in-patients (VA only) and outpatients (both facilities). TTE ordering wasperformed by a group ranging from physician assistants to attendingcardiologists. The overall percentage of classifiable transthoracic echo-cardiographic studies was 84%, similar to the 89% observed by Wardet al4 in a mixed inpatient and outpatient population. Of the classifi-able studies, 91% were appropriate, in keeping with the rates observedin other studies.4,5 Not surprisingly and again consistent with priorstudies, inappropriate transthoracic echo-cardiographic studies wereless common in inpatients than in outpatients (4% vs 10%).

This study represents the first attempt to compare the appropriate-ness of TTE at two types of institutions. VA medical centers oftenresemble academic practices in many respects: trainees may beinvolved in patient care, patients tend to be medically complex, andvolume is often high. Furthermore, faculty members at VA hospitals(including several authors of the article10) often have dual affiliationswith tertiary referral centers. However, VA hospitals are often not asresource rich as academic centers, and the availability of imaging stud-ies may differ between an academic center and its VA affiliate. Itwould be interesting to know how certain logistical factors, such as pa-tient demographics and echocardiography laboratory volume andefficiency, may have affected patterns of TTE ordering in the study.One of the nice features of using a VA hospital is that financial issues,such as reimbursement on the physician side and insurance regula-tions or ability to pay on the patient side, are minimized.

Transthoracic echocardiographic studies ordered by midlevel pro-viders (ie, nurse practitioners and physician assistants) constituteda much larger percentage of the cases in Willens et al’s10 study thanin any previous work. About one third of outpatient studies at theVA were ordered by midlevel providers, and physicians were signifi-cantly more likely to order appropriately. However, no midlevelproviders ordered transthoracic echocardiographic studies at theacademic center, so we cannot generalize this result to non-VA settings.

It is intriguing and reassuring that physician-requested outpatientTTE was equally likely to be appropriate at the university hospitaland the VA hospital, despite the fact that attending cardiologists

requested nearly half of university hospital studies but very few atthe VA hospital. House officers and fellows ordered a large numberof inpatient studies, but very few of the outpatient studies. At bothsites, noncardiology attending physicians in internal medicine andother specialties requested the majority of transthoracic echocardio-graphic studies. No significant difference in the appropriateness rateswas noted between cardiologists and noncardiologists. In contrast,Ward et al4 at the University of Chicago found that cardiac subspecial-ists were significantly more likely to order appropriate TTE than otherphysicians. This could be due to differences in sample size: whereasthe current study included TTE performed in < 600 outpatients,Ward et al4 analyzed >1500 studies. However, even 600 is a reason-able sample size for a study, and in this cohort, there was not evena trend toward a difference. It would be interesting to learn whatfactors at the Miami institutions have eliminated the discrepancybetween cardiologists and other physicians. Given the two differentsites of service, the inclusion of inpatients and outpatients, and thespectrum of providers ordering the studies, it would be interestingto know whether all appropriate transthoracic echocardiographicstudies influenced management and whether all inappropriate studiesdid not. Prior studies have shown the provocative finding that evenstudies classified as inappropriate can yield clinically importantfindings.4,6

The study of Willens et al10 has also added to the list of uses of TTEthat are not covered in the current AC. These include some preoper-ative cardiac risk assessments, as well as some heart failure andchronic coronary artery disease scenarios. Although the AC discussrepeat TTE within a year for a variety of conditions, most important,Willens et al10 found that nearly a quarter of their unclassified indica-tions were echocardiographic studies in valve disease cases repeatedat an interval of >1 year.

Internal medicine residents ordered a majority of inpatient VAtransthoracic echocardiographic studies. Given that nearly all inpa-tient studies were appropriate, it seems that medical residents at theVA hospital ordered TTE judiciously. This supports the prior observa-tion that a large majority of inpatient studies ordered by medical res-idents are appropriate.5 However, little is known about residents’ordering habits in the outpatient setting. Because most medical resi-dency programs emphasize inpatient training over ambulatory care,it is probable that residents would have less familiarity with indicationsfor outpatient TTE and might therefore order more inappropriatestudies in outpatients. The focus on internal medicine house officertraining is slowly shifting toward outpatient care, so it will become im-portant to understand how trainees perform in this setting. Certainly,education on the appropriate use of outpatient noninvasive cardio-vascular imaging must be integrated into training.

In the TTE appropriateness studies published to date, the mostcommon classifiable indications have included symptoms with sus-pected cardiac etiologies (including syncope), prior testing concerningfor heart disease, heart failure, murmur or valvular disease, arrhyth-mia, and endocarditis.4-6,10 Although this concordance may haveresulted from similarities among patient populations, it is more likelybecause these indications all represent common problems. No studyhas yet examined the appropriateness of TTE in a community-onlysetting, but it seems unlikely that the most common indications insuch a study would differ from those documented here.

After reviewing the current publications describing the applicationof the AC for echocardiography, one might ask if the rate of inappro-priate TTE noted of about 10% is ‘‘appropriate.’’ Because this rate wasobserved relatively early after the publication of the AC, it likely rep-resents a ‘‘preintervention’’ baseline from academic medical centers

802 Martin and Picard Journal of the American Society of EchocardiographyJuly 2009

and is a good starting point. The challenge now will be to monitorsuch rates and demonstrate that they can be reduced.

WHAT ARE THE NEXT STEPS?

On the basis of the published studies discussed above, one can predictthat we will see other investigations into the implementation of, andadherence to, the AC for echocardiography. Nonetheless, it is time tomove to the next level of investigation in this field. First, this ought toinclude prospective studies in which providers who order echocardio-graphic studies fill out questionnaires or other tools so that the truereasons for test ordering can be understood. Second, these studiesshould be conducted in multiple centers and settings. This could bedone by combining many single-center studies so that in aggregate,the entire spectrum of practices and settings is included. However,large multicenter studies would be preferable and less prone to bias.Third, one may wish to consider establishing different AC for inpa-tients and outpatients, since not only are the reasons for ordering of-ten different, but also the levels of urgency may be perceived to bedifferent. Last, to examine the ordering patterns of all members ofthe health care team, these future studies will need sufficient samplesizes and specific enrollment criteria.

Many of the AC scenarios are written in such a way that it is likelythat a study classified as appropriate will show abnormalities. Forexample, the scenario for infective endocarditis states that TTE is ap-propriate only if a patient has positive blood culture results or a newmurmur, increasing the pretest probability that appropriate TTE willshow vegetations. Therefore, one might suspect that inappropriatestudies would be unlikely to yield abnormal or useful findings. Twoprevious studies have shown that although appropriate TTE is morelikely to show relevant abnormalities, studies classified as inappropri-ate or unclassifiable studies also frequently provide useful informa-tion.4,6 It has always been challenging to design studies examiningwhether echocardiography alters outcomes. From the studies dis-cussed, an additional challenge is also clear: not only to determine ifTTE for certain indications can be categorized as appropriate or inap-propriate but also to learn if the findings alter patient management oroutcomes and to determine if only studies deemed appropriate alteroutcomes or if even some classified as inappropriate or uncertain mayaffect outcomes by providing useful, albeit unexpected, information.

Although the AC were published only 2 years ago, a considerableamount of data are now available to guide revisions. We believe thatthe American College of Cardiology Foundation and the AmericanSociety of Echocardiography should reconvene the writing groupsand determine if additional common scenarios should be added tothe AC for TTE, TEE, and stress echocardiography. Although thelist should remain manageable, some areas clearly should be ad-dressed. One that may be controversial is a reexamination of theword routine. Although routine evaluations for many conditions inwhich there is no change in clinical status are considered an inappro-priate use of repeat echocardiography, the use of the word yearly inparentheses after the term routine may be interpreted to mean thatit is appropriate to do such repeat studies at intervals of >12 months.That was not the intent. The issue here should not necessarily be thetime interval but the clinical status of the patient. Also, for someunlisted scenarios, it may simply require linking them to current sce-narios. For example, rather than listing many scenarios for various pre-operative risk evaluations, the AC writing group can explicitly statethat the scenarios already listed in the document for TTE and TEEmay apply.

To date, the AC provide a blueprint for quality in echocardiographyby focusing on patient selection, and published studies from a varietyof settings tell us how well we are doing. It would be ideal if cardiol-ogists could integrate these lessons into a method to influence the or-dering of echocardiography and to educate the ordering physicians atthe same time. Although developing such a tool might be time con-suming and may take away some autonomy from other providers,we believe that it would be far better than delegating this to external‘‘benefits managers,’’ as occurs for other imaging tests and has beenmentioned for echocardiography. Perhaps by expanding the AC tofill in the gaps recognized so far, and by implementing some very sim-ple test-ordering rules on the basis of the consistently recognizedinappropriate tests, echocardiographers can demonstrate that healthcare dollars are used more wisely and that patient outcomes arenot jeopardized and are, perhaps, improved.

REFERENCES

1. Pearlman AS, Ryan T, Picard MH, Douglas PS. Evolving trends in the use ofechocardiography: a study of Medicare beneficiaries. J Am Coll Cardiol2007;49:2283-91.

2. Douglas PS, Khanderia B, Stainback RF, Weissman NJ. TTE/TEEAppropriateness Criteria Technical Panel; ACCF Appropriateness Crite-ria Working Group. ACCF/ASE/ACEP/ASNC/SCAI/SCCT/SCMR2007 appropriateness criteria for transthoracic and transesophagealechocardiography: a report of the American College of CardiologyFoundation Quality Strategic Directions Committee AppropriatenessCriteria Working Group, American Society of Echocardiography, Amer-ican College of Emergency Physicians, American Society of NuclearCardiology, Society for Cardiovascular Angiography and Interventions,Society of Cardiovascular Computed Tomography, and the Society forCardiovascular Magnetic Resonance. J Am Soc Echocardiogr 2007;20:787-805.

3. Douglas PS, Khanderia B, Stainback RF, Weissman NJ, Peterson ED,Hendel RC, et al. ACCF/ASE/ACEP/AHA/ASNC/SACI/SCCT/SCMR2008 appropriateness criteria for stress echocardiography. J Am CollCardiol 2008;51:1127-47.

4. Ward RP, Mansour IN, Lemieux N, Gera N, Mehta R, Lang RM. Prospec-tive evaluation of the clinical application of the American College ofCardiology Foundation/American Society of Echocardiography appropri-ateness criteria for transthoracic echocardiography. J Am Coll CardiolCardiovasc Imaging 2008;1:663-71.

5. Martin NM, Picard MH. Use and appropriateness of transthoracic echocar-diography in an academic medical center: a pilot observational study. J AmSoc Echocardiogr 2009;22:48-52.

6. Kirkpatrick JN, Ky B, Rahmouni HW, Chirinos JA, Farmer SA, Fields AV,et al. Application of appropriateness criteria in outpatient transthoracicechocardiography. J Am Soc Echocardiogr 2009;22:53-9.

7. Gibbons RJ, Miller TD, Hodge D, Urban L, Araoz PA, Pellikka P, et al.Application of appropriateness criteria to stress single-photon computertomography sestamibi studies and stress echocardiograms in an academicmedical center. J Am Coll Cardiol 2008;51:1283-9.

8. Mansour IN, Lang RM, Furlong KT, Ryan A, Ward RP. Evaluation of theapplication of the ACCF/ASE appropriateness criteria for transesophagealechocardiography in an academic medical center. J Am Soc Echocardiogr2009;22:517-22.

9. Rao GA, Sajnani NV, Kusnetzky LL, Main ML. Appropriate utilization oftransesophageal echocardiography. Am J Cardiol 2009;103:727-9.

10. Willens HJ, Gomez-Marin O, Heldman A, Chakko S, Postel C, Hasan T,Mohammed F. Adherence to appropriateness criteria for transthoracicechocardiography: comparisons between a regional department of Vet-erans Affairs Health Care System and academic practice and betweenphysicians and mid-level providers. J Am Soc Echocardiogr 2009;22:793-9.