Assessing Quality Improvement in Health Care: Theory for … · Assessing Quality Improvement in Health Care: Theory for Practice abstract OBJECTIVES: To review the role of theory

Assessing Quality Improvement in Health Care Theoryfor Practice

abstractOBJECTIVES To review the role of theory as a means to enhance thepractice of quality improvement (QI) research and to propose a novelconceptual model focused on the operations of health care

METHODS Conceptual model informed by literature review

RESULTS To optimize learning across QI studies requires the inte-gration of small-scale theories (middle-range theories theories ofchange) within the context of larger unifying theories We propose thathealth care QI research would benefit from a theory that describes theoperations of health care delivery including the multiplicity of rolesthat interpersonal interactions play The broadest constructs of themodel are entry into the system and assessment and managementof the patient with the subordinate operations of access recognitionassessment and diagnosis and medical decision-making (developinga plan) coordination of care execution of care referral and reassess-ment respectively Interpersonal aspects of care recognize the patientcaregiver as a source of information an individual in a cultural contexta complex human being and a partner in their care Impacts to any andall of these roles may impact the quality of care

CONCLUSIONS Such a theory can promote opportunities for movingthe field forward and organizing the planning and interpretation ofcomparable studies The articulation of such a theory may simulta-neously provide guidance for the QI researcher and an opportunityfor refinement and improvement Pediatrics 2013131S110ndashS119

AUTHORS Lawrence C Kleinman MD MPH FAAPa andDenise Dougherty PhDb

aDepartment of Health Evidence and Policy Mount Sinai School ofMedicine New York New York and bChild Health and QualityImprovement Agency for Healthcare Research and Quality

KEY WORDSquality of health care (measurement) research methods appliedresearch quality improvement (research and practice) healthservices research Donabedian Deming

ABBREVIATIONQImdashquality improvement

wwwpediatricsorgcgidoi101542peds2012-1427n

doi101542peds2012-1427n

Accepted for publication Dec 20 2012

Address correspondence to Lawrence C Kleinman MD MPHFAAP Mount Sinai School of Medicine Department of HealthEvidence and Policy One Gustave Levy Place Box 1077 New YorkNY 10029 E-mail lawrencekleinmanmountsinaiorg

PEDIATRICS (ISSN Numbers Print 0031-4005 Online 1098-4275)

Copyright copy 2013 by the American Academy of Pediatrics

FINANCIAL DISCLOSURE The authors have indicated they haveno financial relationships relevant to this article to disclose

S110 KLEINMAN and DOUGHERTY by guest on June 16 2020wwwaappublicationsorgnewsDownloaded from

There exists an explicit imperative toimprove the quality of health care de-livered to the American public Two re-markable Institute of Medicine reportsTo Err is Human and Crossing theQuality Chasm codified the foundationsfor such work as they chronicled 2decades of research characterizingpervasive deficits in quality12 Sub-sequently 9 years of reports ordered bythe US Congress document large andpersistent problems in quality includingdisparities in quality by race ethnicityand income3 The nation is coming toagreement on a number of aims forhealth care including health care forchildren The overarching aims as firstarticulated by Donald Berwick and col-leagues4 and now adopted by the fed-eral government are better care betterhealth and lower cost Better care isdefined as health care that is safetimely effective efficient equitable andpatient-centered5 In sum the Institute ofMedicine and US Department of Healthand Human Services reports articulatenational goals Recent legislation in-cluding the Childrenrsquos Health InsuranceProgram Reauthorization Act and theAffordable Care Act signal a new dayfor the study and improvement ofquality There is much activityaddressing quality within and aroundAmerican health care in both thepublic and private sectors6ndash14 Forchildren this is most evident in thecollaboration between the Centers forMedicare and Medicaid Services andAgency for Healthcare Research andQuality to develop and guide 7 Centersof Excellence and 2 state Medicaidprograms in the Pediatric QualityMeasures Program

A September 2012 Institute of Medicinereport summarized a flurry of qualityimprovement (QI) activities and pointedto new directions aimed at making theUS health care delivery system more ofa ldquolearning organizationrdquo15 The reportmay be summarized as articulating the

perspective that ldquoTechnology can fixwhatrsquos broken in health carerdquo16 Thisoptimism is consistent with urgentcalls for action that fall short of em-pirical guidance or even a clear path toimprovement One potential path tobring action together with learning canbe found in the form of applied socialscience17 As specifically applied to theenhancement of performance ldquoim-provement sciencerdquo has been de-scribed as the systematic effort to findldquoout how to improve andmake changesin the most effective way examiningthemethods and factors that best workto facilitate quality improvementrdquo14How to move QI forward is an ongoingsubject of discussion18ndash30

This article argues for the centrality ofscience including rigorous theory de-velopment and testing in moving thenationrsquos quality aims forward In partwe build off of work we presented atthe first 2 ldquoAdvancing the Science ofPediatric Quality Improvement Re-searchrdquo conferences in 2011 and201231ndash33 We identify gaps in the cur-rent theory and practice of QI researchand evaluation in health care and tosuggest approaches to closing thosegaps focusing on pediatric QI as ap-propriate We suggest specific designsshould be matched to specific circum-stances for considering health careimprovement and its evaluation Fi-nally we suggest how practical expe-rience can help to build a theory ofapplied QI in health care that can helpresearchers to identify generalizablefindings from QI evaluation and re-search and help the nation achieve itsgoals

QUALITY IMPROVEMENT

A large stream of QI activities in healthcare are derived from the philosophyof total quality management and thework of Edwards Deming and WalterShewhart34 Themodern era of QI in healthcare began with efforts to stimulate

providersrsquo use of clinical practiceguidelines or to help providers meetperformance goals benchmarkedagainst data from performance mea-sures35ndash37 This era began after it be-came clear that merely producingclinical practice guidelines and pas-sively disseminating them was notleading to change Although there aresome compelling examples of suc-cessful QI efforts focused on specificclinical problems at the clinical micro-system level1538 the current emphasisin QI is on broader systemic effortsdesigned to change the environment inwhich providers practice This empha-sis includes concepts such as systemsreengineering39 delivery system trans-formation 6 s LEAN and systemschange2326 We consider QI broadly toconsist of those systematic data-basedactivities that focus on changing theproduction function of health care withan aim toward improving outcomesandor efficiency Although the fullnessof QI may be manifest in iterative cyclesof measurement and improvement weagree with Berwick1840 that the aimsare more important than the purity ofthe term the empirical emergence ofevidence and understanding createsopportunities to iteratively posit andtest conceptual models and theoriesHis other point in 1996 that we need tobroaden our ideas of what constitutesgood science is as relevant today asthen

In 2003 Galvin and McGlynn41 consid-ered 3 social science models ChangeTheory Tipping Point and DiffusionTheory to explore why QI had notresulted in a fundamentally improvedhealth care system They recom-mended a strategy that used perfor-mance measurement to drive changeTen years later observers are stillasking why health care quality haslargely not improved34 and attemptingto develop frameworks to help guideimprovement42

SUPPLEMENT ARTICLE

PEDIATRICS Volume 131 Supplement 1 March 2013 S111 by guest on June 16 2020wwwaappublicationsorgnewsDownloaded from

A Science of QI Theory for Practice

We are not the first to suggest the im-portanceof science and theory inQI43ndash45

The journal Implementation Sciencewhich publishes much of the researchin QI now recommends thatmanuscriptsubmitters articulate their theory ofchange46 Duncan Neuhauser a distin-guished expert in medicine and healthcare organization recently summarizedhow theory development fits with theepistemology of knowledge and thescientific method as well as the im-portance of theory in QI47 He remindsus that ldquothe scientific method consistsof theory development making pre-dictions (hypotheses) hypothesis test-ing theory revisionhelliprdquo (p 466) andnotes that ldquoTheory is essential becauseit allows us to generalize to other placestimes and circumstanceshellipTheory isthe way we learn from each otherhelliprdquo

(p 466) This approach is a clear de-scendant of Mertonrsquos seminal vision ofmiddle range theory48

Amongcharacteristicsof goodQI theorythat Neuhauser and his colleaguessuggest is the specification of variablesand relationships among variables useof outcomemeasures related to qualitynumerical measurement of variablesand hypothesis testing (see Appendix)Although there may be disagreementabout the defining characteristics ofgood theory in general even thosesuggested by Neuhauser are almostnever found in QI work no less in QIresearch publications in the scientificliterature394549ndash51 Thus much of QI isa ldquoblack boxrdquo52 providing little guidanceto those eager to take a more informedapproach to improving care and cast-ing doubt on its value as a science Un-derstanding substantive interactionsbetween specified aspects of inter-action and identifiable attributes ofcontext may be critical to support in-telligent generalization from one QIproject to the next another reason tointegrate theory into this work53

The imperative to develop and test theoryin QI may sound daunting to those whohave not been trained in the social andbehavioral sciences where theory de-velopment and hypothesis testing aremundane The word should not in-timidate theory may be a well-specifiedconceptual model that predicts andexplains what the intervention is whatneed it is filling what is supposed tohappen as a result (the outcome) howand why the results happened on aver-age and optimally reasons for varia-tions across sites in amultisite strategyEvaluation scientists might label sucha theory a logic model with a theory ofchange Many theories are available touseasstartingpoints inQIor theycanbecombined in complementary ways Theimportant point is to carefully thinkthrough (and specify ahead of time) therelationships among variables such asldquoA leads to B controlling for (or ac-counting for) Chelliprdquo 435455

Research Methods for QI

Randomized controlled trials with thepatient as the unit of intervention areoften not optimal for health care QIstudies The unit of intervention is typ-ically the organization (eg clinicalpractice health plan state) although itmay be the individual provider Clusterrandomized controlled trials would beoptimal and have been conductedprimarily in other countries56ndash60 In theUnited States randomization of pro-viders plans and other QI locations isalmost unheard of61ndash63 Rigorous quasi-experiments or observational studies(eg interrupted time-series regres-sion discontinuity) preferably usingcomparison groups have been rec-ommended64 and will likely increaseas a result of recent efforts to trainmore researchers in their use Eventhe use of contemporaneous nonequiv-alent control group designs wouldrepresent significant improvementover the dominant use of post-only

or pre-post studies without controlgroups

The understanding of context on theresults of QI interventions is essential tobuilding this science and making ituseful to frontline implementers Associal psychologists have demon-strated for more than 60 years be-havior (and outcomes) is a function ofthe person (eg the provider) in his orherenvironment Although thispremiseis now accepted in the QI world a trueunderstanding of context and an abilityto test the impact of contextual varia-bles on improvement goals is not likelyto occur without development andmore common use of standardizeddefinitions and instruments For ex-ample the following are among themost common features hypothesized(mostly post hoc) to account for vari-ation in the results of performanceimprovement strategies leadershipquality of team work collective mind-fulness65 resources organizationalcultures nature of hierarchies andcommunications styles These are oftentaken as an article of faith but the evi-dence base for them is highly variable inpart because they are either undefinedor defined differently in every studyHealth care experts have been skepticalabout the value of using concepts andinstruments from other fields nonethe-less starting points outside of the healthcare context can fruitfully guide the de-velopment of health carendashspecific toolsOnce themeasures are defined they canthen be used to study the heterogeneityof effects in QI66 Using a biologicalmetaphor interventions have activesites and organizations have receptorsand a mismatch therein can cause anotherwise effective intervention to fail ina specific context Moreover the effec-tiveness of an intervention may dependon the need manifest in the organiza-tional context immunizing already pro-tected individuals will confer noadditional benefit Needs analyses may


identify gaps in resources or opportu-nities to improve processes that may beturned into targets for directed QI ini-tiatives We must develop ways to dealproductively in contexts replete withuncertainties

Improving the science of improvementwill takeconcertedeffortand increasedthoughtfulness in the clinical andhealth services research communitiesSome entities have begun this processincluding in the United States the Ac-ademic Pediatric Association the Ac-ademy for Healthcare Improvementand the Improvement Science ResearchNetwork In theUnitedKingdom thenewHealth Foundation is takingmajor stepsto build the science and contribute toachievingbettercarebetterhealthandlowercosts Others have suggested thata new international entity be formed toaccelerate progress reactions to thisidea have been mixed perhaps be-cause new organization may be seen tothreaten rather than complement exist-ing ones67

Still others are impatient with calls toimprove and use improvement scienceunderstandably focusing on an urgentcall for action However action andtheory are not incompatible as it hasbeen said there is nothing so practicalas a good theory68 Indeed ldquoaction re-searchrdquo is defined as ldquocomparativeresearch on the conditions and effectsof various forms of social action andresearch leading to social actionrdquo us-ing iterative cycles of planning actionand measurement69 Action researchmay be valuable in health care QI andother systems change research andoffers significant promise as 1 foun-dation for intelligent change70ndash73

Moving Forward Toward a Model forQuality Improvement

We suggest a need to progress on 2intersecting fronts empirical workneeds to document and improve care inreal time and stakeholder engagement

to develop a meaning-based under-standing of health care

Structure process and outcome havebeen central to the theory of health carequality since Donabedian articulatedan architecture for measuring healthcare quality and laid the foundation forthe recent explosion of work in thefield We call on the development ofnew theory to complement Dona-bedianrsquos work to advance from hisstrict focus on measurement and to-ward health care operations and im-provement By focusing on the tasks ofhealth care in context we may come tolearn about what makes a difference inhealth and health care An operations-based model supports QI activities asit elicits relationships between rootcauses and outcomes

Such a model could then generatemiddle-range theories in the form oftheories of change These theoriescould be used to drive the designand implementation of interventionsdesigned to improve quality Such the-ories would be testable and supportthe modification of the overall theory

One can easily imagine how the de-velopment of such theory could beleveraged to make the Plan-Do-Check-Act cycles of QI both more efficient(as it can drive the Plan and Do com-ponents) and more informative Sucha theory has the potential to simulta-neously limit degrees of freedom in QIactivity and challenge those working inthe field to link their work in individualinstitutions to ideas that transcendsuch boundaries7475 This theory ofpractice could suggest a taxonomy ofthe work of health care in relativelyfundamental terms that describe con-structs that can both be measured andmodified Such a model could bothsupport and rely on theories of changeto advance an epistemology of QI ap-propriate to the health care context Anoperations-based model of quality maychallenge those of us struggling to

improve quality to refine and developwhat is proposed here in a way thatenhances our ability to understandand effect the production of qualityhealth care We set a toe in the waterhere

Current Theory

According to Donabedian the structureof care relates to the organization ofhealth care delivery whether at thelevel of the health care system the or-ganizational or corporate unit or theindividual practice7071 An example ofa structural defect in health care is thelack of accessibility of care to thosewho are unable to afford it On the mi-cro level the inability of a patient toaccess a physicianrsquos office that wasnot wheelchair accessible also wouldrepresent a structural impedimentSimilarly the organization of qualitymeasurement data infrastructure andthe nature of quality assessment orimprovement activities are all struc-tural aspects of care The process ofcare includeswhat is done and not doneto or for the patient or by members ofthe health care system Traditionallythis was divided into 2 categories in-terpersonal and technical The in-terpersonal aspects of process relatedto how a patient was treated on a hu-man level was the encounter with thehealth care system characterized byrespect for the patient his or her needsdesires and privacy and so forth Thetechnical aspects of health care canfurther be divided into medical decision-making and technical skill in imple-menting medical decisions Brook andcolleagues introduced the construct ofappropriateness and necessity of carebased on the relative balance of riskbenefits and importance76ndash80 The recentfield of patient safety has emerged tofocus on the technical quality of carespecifically related tomedical errors238182

The failure to deliver a planned or ne-cessary service represents anothertype of medical error as does failure to

SUPPLEMENT ARTICLE


notify or provide appropriate follow-upas might occur after an abnormalcervical cancer screen is victimized bya medical error

It is axiomatic that in aggregate healthcare impacts health that its processesvariably impact outcomes Process isinherently more sensitive to measurechange than are outcomes Processmeasurement suffers in practice be-cause of uncertainty in the link betweenspecific processes and specific out-comes Uncertainty may reflect theactual relationship between processand outcomes the measurement chal-lenge or the research effort (eg farmore research links processes to out-comes in adults than in children)83

Outcomes of health care are defined aswhat actually becomes of the patientand may represent ultimate con-sequences the degree of health or ill-ness or mortality or intermediateaspects of care such as the need for anemergencydepartment visit Ingeneralthe better the evidence linking the in-termediate outcome to the ultimateoutcomes (or the more important theintermediate outcome is considered inand of itself) the more confident onecan be using intermediate outcomes todescribe the quality of care

Because outcomes are the most visibleand intuitive component of qualitythere has been a large move towardemphasizing outcomes in policy deci-sions When doing so one must rec-ognize that because of the resiliency ofindividuals and populations and be-cause of the indefinite relationshipbetween processes and outcomesoutcomes are an insensitive way tomeasure changes in the quality of careFor example only rarely does a childwho is not fully immunized suffer froma vaccine-preventable illness yet thefailure to fully immunize each childrepresentsadecrement in thequality ofcare This problem is amplified whenoutcomes are uncommon

Operations-based Model

The improvement of health care qualityis a dynamic process and QI is just nowbeing defined rigorously in the litera-ture84 We propose that the work ofhealth care can be defined in an oper-ational and interpersonal frameworkto underlie QI research in much thesame way that Donabedian definesquality research Figure 1 illustratesthat either through the action of anindividual patient or the health caresystem there emerges an awarenessof a need or opportunity for healthcare Subsequent to such awarenessaccess is or is not achieved Once careis accessed there comes an assess-ment of the extent to which there isa concern a need or an opportunitySuch conditions result for examplefrom a symptom (CONCERN) a treat-able illness (NEED) or a circumstance(eg underimmunized 15-month-oldchild representing an OPPORTUNITY)The concern need or opportunity thenneeds to be assessed and defined bythe clinician or health care system (egthe provision of visit reminders forpreventive health care in a proactivemanaged care plan) A preliminaryassessment is confirmed by history orphysical and a diagnosis may be made(the ldquodiagnosisrdquo in this respect mayrepresent a confirmation of the need toundertake further evaluation) A fur-ther care management plan may benegotiated and implemented with co-ordination and execution of processesof care We note that this care man-agement may be for further diagnosticwork treatment follow-up or somecombination of those At times referralmight represent an auxiliary or a newmain path reinitiating these con-structs With new information comesthe need to reassess recognition ofa concern need or opportunity orpotentially the firm recognition of theidentification of actionable information(even the recognition of uncertainty

that requires action) may restart thecycle In this framework diagnosis ofa state occurs when the recognitionbecomes actionable Actions may rep-resent large activities or smaller itera-tive data-gathering ones The broadestconstructs of the model are entry as-sessment and management with thesubordinate operations of access rec-ognition assessment and diagnosisandmedical decision-making (developinga plan) coordination of care executionof care referral and reassessmentrespectively

Figure 2 breaks down the various in-terpersonal components involved inproviding health care Interpersonaltasks include eliciting relevant clinicalinformation from the patient or familyindividualizing care plans with respectfor patient values and recognizinglife contexts (operationalized by manyin the construct of shared decision-making) developing a relationshipthat optimizes the healing power of thetherapeutic interpersonal relationshipand developing a partnership with thepatient that reinforces other aspects ofrespectful behavior in a way that canbe operationalized as a therapeuticpartnership

We articulate this model as a strawperson to initiate specific discussionregarding how to reframe quality notonly as an object for measurement aswas done so ably by Donabedian but asa target for improvement We presentthis theory too as an illustration of thepotential to articulate paradigmatictheories that are overarching to themiddle-range theories described in thisarticle

CONCLUSIONS

In someways growth and developmentin pediatrics provides an apt metaphorfor QI research There is movementthrough time and change some in-tended some not and some inevitabledespite all attempts to steer it


otherwise These are systems beyondour control but not beyond our di-rection Understanding how to increasethe desired outcome when raisinga child requires some understanding ofhow children grow what influencesthem and where they are heading Itrequires a theory that accounts for thedynamic nature of being a child ina family in a community and the mul-titude of potential influencers Thena focus on those influences seen as keymay help a parent to raise a healthywell-adjusted and productive youngadult

QI research in general and particularlythat for children requires an apprecia-tionof the impactandchangesover timein terms of the structures processes

and outcomes of care in the immedi-ate future and across the life courseEvidence is a critical foundation forsuch work but so are reasonabletheories articulated and specified fortesting

We suggest that the task of researchersworking to build a thriving field of QIresearch includes the following

Developing and extending the the-ory of health care operations anddelivery through systematic in-quiry that incorporates both quan-titative and qualitative methods Anideal theory will allow for the as-sessment of care both to an indi-vidual patient and to populationsThe Institute of Medicine suggests7 attributes of quality including

equity that should be accountedfor at the population level

Integrating theory and empiricalfindings to identify key variablesthat affect the quality of healthcare We expect that some of thesevariables will be clinical processvariables others structural varia-bles (which may be as varied assystems for ongoing training orga-nizational attributes and access totechnology) and operational pro-cess variables As theory identifiesa causal chain some structuraland process variables may emergeas sufficiently important to serveas outcomes in their own rightor as proxies for clinical or otheroutcomes Longer-term clinical

FIGURE 1An operational approach to organizing health care Major tasks include entry into the health care system assessment of needs and opportunities (includingdiagnosis) and management including medical decision-making execution and coordination and follow-up

SUPPLEMENT ARTICLE


outcomes should be identified andmay be monitored for trends acrossthe body of change that is accom-plished

Develop valid and reliable measuresthat permit the assessment of keyvariables in a variety of settings

Develop and assess tools andmethods that improve the abilityof researchers and evaluators ofQI work to grow the field An im-proved understanding of the episte-mology of QI research and evaluationwould be a critical componentto both guide and assess theirdevelopment

Develop and evaluate policies thatpromote the conversion of infor-mation and understanding abouthealth care QI from a private goodto a public good8586 Althoughthere has been a good deal of at-tention paid to the ldquobusiness casefor qualityrdquo much less attention

has been paid to how to developsufficient value for private healthcare organizations that share ratherthan hoard information data andunderstanding Public funds shouldsupport only QI work that is de-signed to provide information beyondunderstanding whether a particularintervention worked in a particularorganizational setting

Develop and evaluate interventionsthat social science organizationalor other theory suggest may holdpromise to improve the quality ofcare In developing an understand-ing of the impact of interventions itwill be important to assess attribu-tion as well as association Excellentdesign triangulation through theuse of complementary designs andcareful analysis including thethoughtful incorporation of BradfordHillrsquos classic principles may guide usin developing an understanding of

the cause and effect of QI interven-tions Attention to context may re-quire attention to closing gaps(additive models) changing pro-cesses (multiplicative models) orboth

We have articulated a theory of qualityas a target for improvement that de-scribes clinical and interpersonal tasksthat constitute our initial framing of thework of health care These tasks occurwithin clinical organizational andcommunity contexts that may requirerecognition of the ldquoreceptor sitesrdquo thatdefine how those contexts will modifythe meaning or measurement of thesevarious operations in that contextRecognizing the active components ofcontext on QI and how they interactwith these operations will representa fundamental task for the QI re-searcher Nonetheless by focusing onpotentially observable operations ofhealth care this or similar frameworks

FIGURE 2Characterizing the various roles that patientsparentscaregivers may play that are developed through the interpersonal aspects of health care delivery Thetask of the clinicianhealth care system is to cultivate each role in appropriate measure


offer the potential to build on theDonabedian framework to support and

extend the conceptual model from the

more static construct of quality mea-

surement to the more dynamic con-

struct of QI research

Quality and QI research require thecollaboration of excellent listeners so-phisticated methodologists and con-ceptual and analytic thinkers allcoming together with open minds thatare informedbutnot boundby theory Inthis effort the need to create a para-digmfor that incorporates thebenefit ofmultiple disciplines in pursuit of betterhealth care can be achieved Whenconsidering the quality of health carefor children additional complexitiesare added as is the opportunity to in-corporate a fundamental appreciation

for the importance of the life course forchildrenrsquos total health In this effort theneed to create a paradigm for thatincorporates the benefit of multipledisciplines in pursuit of better healthcare can be achieved

APPENDIX

1 The variables and their relation-ships are clearly defined

2 The resultant model or theory isgeneralizable to other organiza-tions

3 The theory leads to predictionsthat can potentially be disproved

4 The theory leads to the possibilityof replication

5 The theory simplifies reality Sim-plicity is a virtue

6 The theory predicts better thanother theories

7 Numerical measurement of varia-bles and their relationships arenot required but they are usefulowing to their preciseness

8 The unit of analysis is defined In[quality improvement] the unit ofanalysis is [often] the organization[or the provider] In clinical re-search the unit of analysis is oftenthe patient This has important re-search consequences

9 The dependent outcome variable isrelated to quality such as fewererrors lower mortality lower costsand patient satisfaction ldquoThis out-come variable is what makes it atheory of health care quality improve-mentrdquo [emphasis added]43(pp 466-467)

REFERENCES

1 Institute of Medicine To Err is HumanBuilding a Safer Health System WashingtonDC National Academy Press 1999

2 Institute of Medicine Crossing the QualityChasm A New Health System for the 21stCentury Washington DC National AcademyPress 2001

3 Agency for Healthcare Research and Qual-ity Measuring health care quality Availableat wwwahrqgovqualmeasurixhtmAccessed December 1 2012

4 Berwick DM Nolan TW Whittington J Thetriple aim care health and cost Health Aff(Millwood) 200827(3)759ndash769

5 Institute of Medicine Crossing the QualityChasm Washington DC National AcademyPress 2001

6 Agency for Healthcare Research and Qual-ity CHIPRA childrenrsquos health care qualitymeasurement and improvement activitiesAvailable at wwwahrqgovchipra AccessedDecember 1 2012

7 Centers for Medicare amp Medicaid ServicesMedicaid CHIP program information Qual-ity of Care Available at wwwmedicaidgovMedicaid-CHIP-Program-InformationBy-TopicsQuality-of-CareQuality-of-Carehtml AccessedDecember 1 2012

8 Agency for Healthcare Research and QualityThe national strategy for quality improve-ment in health care Available at wwwahrq

govworkingforquality Accessed December1 2012

9 Khuri SF Henderson WG Daley J et alPrincipal Investigators of the Patient Safety inSurgery Study Successful implementation ofthe Department of Veterans Affairsrsquo NationalSurgical Quality Improvement Program in theprivate sector the Patient Safety in Surgerystudy Ann Surg 2008248(2)329ndash336

10 New York State Department of Health Pa-tient safety center Related publicprivateinitiatives Available at wwwhealthnygovprofessionalspatientspatient_safetyAccessed December 1 2012

11 WellPoint Inc Press release CMS adoptsprivate sector primary care solution toimprove quality and reduce medical costsAvailable at httpirwellpointcomphoenixzhtmlc=130104ampp=irol-newsArticleampID=1703173amphighlight= (CMS adopts privatesector solution to improvehellip) AccessedSeptember 5 2012

12 Improvement Science Research NetworkWhat is improvement science Available athttpisrnnetaboutimprovement_scienceasp Accessed December 1 2012

13 USAID Health Care Improvement ProjectThe science of improvement Available at2012wwwhciprojectorgimprovement_toolsimprovement_methodsscience AccessedDecember 1

14 The Health Foundation Evidence scan Im-provement science Available at wwwhealthorgukpublicationsimprovement-scienceAccessed December 2 2012

15 Institute of Medicine Best Care at LowerCost The Path to Continuously LearningHealth Care in America Washington DCNational Academy Press 2012

16 AHIP Solutions SmartBrief [ahipsmart-briefcom] September 7 2012

17 Freeman HE Rossi PH Furthering the ap-plied side of sociology Am Sociol Rev 198449(4)571ndash580

18 Berwick DM The question of improvementJAMA 2012307(19)2093ndash2094

19 McCannon J Berwick DM A new frontier inpatient safety JAMA 2011305(21)2221ndash2222

20 Clancy CM Berwick DM The science ofsafety improvement learning while doingAnn Intern Med 2011154(10)699ndash701

21 Pellegrin KL Carek D Edwards J Use of ex-perimental and quasi-experimental methodsfor data-based decisions in QI Jt Comm JQual Improv 199521(12)683ndash691

22 Chassin MR Quality of care how good isgood enough Isr J Health Policy Res 20121(1)4

23 Pronovost PJ Bo-Linn GW Preventing pa-tient harms through systems of care JAMA2012308(8)769ndash770

SUPPLEMENT ARTICLE


24 Bielaszka-DuVernay C Improving qualityand safety Available at wwwhealthaffairsorghealthpolicybriefsbriefphpbrief_id=45 Accessed December 1 2012

25 Croxson B Hanney S Buxton M Routinemonitoring of performance what makeshealth research and development different JHealth Serv Res Policy 20016(4)226ndash232

26 Chassin MR Loeb JM The ongoing qualityimprovement journey next stop high re-liability Health Aff (Millwood) 201130(4)559ndash568

27 Leatherman ST Hibbard JH McGlynn EA Aresearch agenda to advance quality mea-surement and improvement Med Care200341(suppl 1)I80ndashI86

28 McGlynn EA Cassel CK Leatherman STDeCristofaro A Smits HL Establishing na-tional goals for quality improvement MedCare 200341(suppl 1)I16ndashI29

29 Khodyakov D Hempel S Rubenstein L et alConducting online expert panels a feasibil-ity and experimental replicability studyBMC Med Res Methodol 201111174

30 Rubenstein LV Hempel S Farmer MM et alFinding order in heterogeneity types ofquality-improvement intervention pub-lications Qual Saf Health Care 200817(6)403ndash408

31 Kleinman LC Dougherty D Health care qualityimprovement research theory and its appli-cation in statistical analysis Paper presentedat Pediatric QI Methods Research an Eval-uation Conference April 29 2011 Denver COAvailable at wwwacademicpedsorgspecial-InterestGroupsQIfolderKleinman_Doughertypdf Accessed December 2 2012

32 Kleinman L Welcome and overview pre-sentation Paper presented at 2nd AnnualAdvancing Quality Improvement Science forChildrenrsquos Healthcare Research ConferenceApril 27 2012 Boston MA Available at wwwacademicpedsorgspecialInterestGroupsQIfolder2012Kleinman_BWelcome_Overviewpdf Accessed December 1 2012

33 Kleinman L Risk adjustment for QI Re-search 101 Paper presented at 2nd AnnualAdvancing Quality Improvement Science forChildrenrsquos Healthcare Research ConferenceApril 27 2012 Boston MA Available at wwwacademicpedsorgspecialInterestGroupsQIfolder2012Kleinman_BWelcome_Overviewpdf Accessed December 1 2012

34 Deming WE Out of the Crisis CambridgeMA MIT Press 2000

35 Luce JM Bindman AB Lee PR A brief his-tory of health care quality assessment andimprovement in the United States West JMed 1994160(3)263ndash268

36 Chassin MR OrsquoKane ME History of thequality improvement movement In Berns

SD ed Toward Improving the Outcome ofPregnancy III Enhancing Perinatal HealthThrough Quality Safety and PerformanceInitiatives March of Dimes Web site De-cember 2010 Available at wwwmarch-ofdimescomTIOPIII_finalmanuscriptpdfAccessed December 2 2012

37 Nigam A Changing health care qualityparadigms the rise of clinical guidelinesand quality measures in American medi-cine Soc Sci Med 201275(11)1933ndash1937

38 McKethan A Shepard M Kocot SL et alImproving quality and value in the UShealth care system Bipartisan Policy Cen-ter August 2009 Available at httpbipar-tisanpolicyorglibraryreportimproving-quality-and-value-us-health-care-systemAccessed December 1 2012

39 Sanchez JA Barach PR High reliabilityorganizations and surgical microsystemsre-engineering surgical care Surg ClinNorth Am 201292(1)1ndash14

40 Berwick DM Harvesting knowledge fromimprovement JAMA 1996275(11)877ndash878

41 Galvin RS McGlynn EA Using performancemeasurement to drive improvement a roadmap for change Med Care 200341(suppl 1)I48ndashI60

42 Kaplan HC Provost LP Froehle CM MargolisPA The Model for Understanding Successin Quality (MUSIQ) building a theory ofcontext in healthcare quality improvementBMJ Qual Saf 201221(1)13ndash20

43 Hulscher ME Schouten LM Grol RP BuchanH Determinants of success of quality im-provement collaboratives what does theliterature show [published online ahead ofprint August 9 2012] BMJ Qual Saf

44 Dixon-Woods M Bosk CL Aveling EL GoeschelCA Pronovost PJ Explaining Michigan de-veloping an ex post theory of a quality im-provement program Milbank Q 20189(2)167ndash205

45 Scott A Sivey P Ait Ouakrim D et al Theeffect of financial incentives on the qualityof health care provided by primary carephysicians Cochrane Database Syst Rev2011(9)CD008451

46 Eccles MP Foy R Sales A Wensing MMittman B Implementation Science sixyears onmdashour evolving scope and commonreasons for rejection without review Im-plement Sci 2012771

47 Best M Neuhauser D Did a cowboy rodeochampion create the best theory of qualityimprovement Malcolm Baldrige and hisaward BMJ Qual Saf 201120(5)465ndash468

48 Merton RK On sociological theories of themiddle range In Merton RK ed Social Theoryand Social Structure New York NY Simon ampSchuster the Free Press 194939ndash53

49 Shekelle PG Pronovost PJ Wachter RMet al Advancing the science of patient safetyAnn Intern Med 2011154(10)693ndash696

50 Kaplan HC Brady PW Dritz MC et al Theinfluence of context on quality improve-ment success in health care a systematicreview of the literature Milbank Q 201088(4)500ndash559

51 Oslashvretveit J Understanding the conditionsfor improvement research to discoverwhich context influences affect improve-ment success BMJ Qual Saf 201120(suppl1)i18ndashi23

52 Broer T Nieboer AP Bal RA Opening theblack box of quality improvement collabo-ratives an Actor-Network theory approachBMC Health Serv Res 201010265

53 Kleinman LC Eder M Hacker K for theContext Subcommittee of the OutcomesCommittee of the Community EngagementKey Function Committee of the NationalCTSA Consortium White paper encouragingan agenda for social behavioral and eco-nomic sciences to advance measurementserving community based research Sub-mitted to the National Science Foundation aspart of its SBE 2020 planning activity Availableat wwwnsfgovsbesbe_20202020_pdfsKleinman_Lawrence_224pdf Accessed De-cember 1 2012

54 Glanz K Rimer BK Viswanath K HealthBehavior and Health Education Theory Re-search and Practice 4th ed San FranciscoCA Jossey-Bass 2008

55 DiClemente RJ Crosby RA Kegler MCEmerging Theories in Health PromotionPractice and Research 2nd ed San FranciscoCA Jossey-Bass 2009

56 Huis A Schoonhoven L Grol R Donders RHulscher M van Achterberg T Impact ofa team and leaders-directed strategy toimprove nursesrsquo adherence to hand hy-giene guidelines a cluster randomisedtrial [published online ahead of print Au-gust 28 2012] Int J Nurs Stud

57 Drewelow E Wollny A Pentzek M et al Im-provement of primary health care of patientswith poorly regulated diabetes mellitus type2 using shared decision-makingmdashthe DE-BATE trial BMC Fam Pract 201213(1)88

58 Young D Furler J Vale M et al PatientEngagement and Coaching for Health ThePEACH studymdasha cluster randomised con-trolled trial using the telephone to coachpeople with type 2 diabetes to engage withtheir GPs to improve diabetes care a studyprotocol BMC Fam Pract 2007820

59 Berwanger O Guimaratildees HP Laranjeira LNet al Bridge-Acs Investigators Effect of amultifaceted intervention on use of evidence-based therapies in patients with acute


coronary syndromes in Brazil the BRIDGE-ACS randomized trial JAMA 2012307(19)2041ndash2049

60 Scales DC Dainty K Hales B et al A mul-tifaceted intervention for quality improve-ment in a network of intensive care unitsa cluster randomized trial JAMA 2011305(4)363ndash372

61 Shafer MA Tebb KP Pantell RH et al Effectof a clinical practice improvement inter-vention on Chlamydial screening amongadolescent girls JAMA 2002288(22)2846ndash2852

62 Holmboe ES Hess BJ Conforti LN Lynn LAComparative trial of a web-based tool toimprove the quality of care provided toolder adults in residency clinics modestsuccess and a tough road ahead AcadMed 201287(5)627ndash634

63 Stout JW Smith K Zhou C et al Learningfrom a distance effectiveness of onlinespirometry training in improving asthmacare Acad Pediatr 201212(2)88ndash95

64 Mercer SL DeVinney BJ Fine LJ Green LWDougherty D Study designs for effectivenessand translation research identifying trade-offs Am J Prev Med 200733(2)139ndash154

65 Weick KE Sutcliffe KM Managing the Un-expected Resilient Performance in an Ageof Uncertainty San Francisco CA Jossey-Bass 2007

66 Duan N Description of ideal evaluationmethods quantitative approaches to con-text heterogeneity In Assessing the Evi-dence for Context-Sensitive Effectivenessand Safety of Patient Safety Practices De-veloping Criteria Contract Final ReportAgency for Healthcare Research and Qual-ity December 2010 Available at wwwahrqgovqualcontextsensitivecontext12htmAccessed December 1 2012

67 Wensing M Grimshaw JM Eccles MP Doesthe world need a scientific society for re-search on how to improve healthcareImplement Sci 2012710 Available at wwwimplementationsciencecomcontent7110Accessed December 1 2012

68 Lewin K Field Theory in Social Science NewYork NY Harper 1952

69 Lewin K Action research and minorityproblems J Soc Issues 19462(4)34ndash46

70 Burns D Systemic Action Research AStrategy for Whole System Change BristolUK Policy Press 2007

71 Kirschner K Braspenning J Jacobs JE GrolR Design choices made by target users fora pay-for-performance program in primarycare an action research approach BMCFam Pract 20121325

72 Green LW Making research relevant if it isan evidence-based practice wherersquos thepractice-based evidence Fam Pract 200825(suppl 1)i20ndashi24

73 Green LW Glasgow RE Evaluating the rele-vance generalization and applicability ofresearch issues in external validation andtranslation methodology Eval Health Prof200629(1)126ndash153

74 Donabedian A Definition of Quality andApproaches to Its Assessment (Explorationsin Quality Assessment and Monitoring Vol 1)Chicago IL Health Administration Press1980

75 Donabedian A The Criteria and Standardsof Quality (Explorations in Quality As-sessment and Monitoring Series Vol 2)Chicago IL Health Administration Press1982

76 Brook RH Chassin MR Fink A Solomon DHKosecoff J Park RE A method for the de-tailed assessment of the appropriateness

of medical technologies Int J Technol As-sess Health Care 19862(1)53ndash63

77 Kahn KL Kosecoff J Chassin MR et alMeasuring the clinical appropriateness ofthe use of a procedure Can we do it MedCare 198826(4)415ndash422

78 Park RE Brook RH Appropriateness stud-ies N Engl J Med 1994330(6)432ndash authorreply 433ndash434

79 Kahan JP Bernstein SJ Leape LL et alMeasuring the necessity of medical pro-cedures Med Care 199432(4)357ndash365

80 Brook RH Assessing the appropriatenessof caremdashits time has come JAMA 2009302(9)997ndash998

81 Reason JT Hobbs A Managing Mainte-nance Error A Practical Guide AldershotUK Ashgate Publishing Ltd 2003

82 Reason JT The Human Contribution UnsafeActs Accidents and Heroic RecoveriesAldershot UK Ashgate Publishing Ltd 2008

83 Kleinman LC Prevention and primary careresearch for children The need for evi-dence to precede ldquoevidence-basedrdquo Am JPrev Med 199814(4)345ndash351

84 OrsquoNeill SM Hempel S Lim YW et al Identi-fying continuous quality improvementpublications what makes an improvementintervention lsquoCQIrsquo BMJ Qual Saf 201120(12)1011ndash1019

85 Kleinman LC Kosecoff J Dubois RW BrookRH The medical appropriateness of tym-panostomy tubes proposed for childrenunder 16 years in the United States JAMA19942711250ndash1255

86 Kleinman LC Boyd EA Heritage JC Adher-ence to prescribed explicit criteria duringutilization review An analysis of communi-cations between attending and reviewingphysicians JAMA 1997278(6)497ndash501

SUPPLEMENT ARTICLE


DOI 101542peds2012-1427n2013131S110Pediatrics

Lawrence C Kleinman and Denise DoughertyAssessing Quality Improvement in Health Care Theory for Practice

ServicesUpdated Information amp

httppediatricsaappublicationsorgcontent131Supplement_1S110including high resolution figures can be found at

References

BIBLhttppediatricsaappublicationsorgcontent131Supplement_1S110This article cites 50 articles 7 of which you can access for free at

Subspecialty Collections

subhttpwwwaappublicationsorgcgicollectionquality_improvement_Quality Improvemente_management_subhttpwwwaappublicationsorgcgicollectionadministrationpracticAdministrationPractice Managementfollowing collection(s) This article along with others on similar topics appears in the

Permissions amp Licensing

httpwwwaappublicationsorgsitemiscPermissionsxhtmlin its entirety can be found online at Information about reproducing this article in parts (figures tables) or

ReprintshttpwwwaappublicationsorgsitemiscreprintsxhtmlInformation about ordering reprints can be found online

by guest on June 16 2020wwwaappublicationsorgnewsDownloaded from



httppediatricsaappublicationsorgcontent131Supplement_1S110located on the World Wide Web at

The online version of this article along with updated information and services is

ISSN 1073-0397 60007 Copyright copy 2013 by the American Academy of Pediatrics All rights reserved Print the American Academy of Pediatrics 141 Northwest Point Boulevard Elk Grove Village Illinoishas been published continuously since 1948 Pediatrics is owned published and trademarked by Pediatrics is the official journal of the American Academy of Pediatrics A monthly publication it


There exists an explicit imperative toimprove the quality of health care de-livered to the American public Two re-markable Institute of Medicine reportsTo Err is Human and Crossing theQuality Chasm codified the foundationsfor such work as they chronicled 2decades of research characterizingpervasive deficits in quality12 Sub-sequently 9 years of reports ordered bythe US Congress document large andpersistent problems in quality includingdisparities in quality by race ethnicityand income3 The nation is coming toagreement on a number of aims forhealth care including health care forchildren The overarching aims as firstarticulated by Donald Berwick and col-leagues4 and now adopted by the fed-eral government are better care betterhealth and lower cost Better care isdefined as health care that is safetimely effective efficient equitable andpatient-centered5 In sum the Institute ofMedicine and US Department of Healthand Human Services reports articulatenational goals Recent legislation in-cluding the Childrenrsquos Health InsuranceProgram Reauthorization Act and theAffordable Care Act signal a new dayfor the study and improvement ofquality There is much activityaddressing quality within and aroundAmerican health care in both thepublic and private sectors6ndash14 Forchildren this is most evident in thecollaboration between the Centers forMedicare and Medicaid Services andAgency for Healthcare Research andQuality to develop and guide 7 Centersof Excellence and 2 state Medicaidprograms in the Pediatric QualityMeasures Program

A September 2012 Institute of Medicinereport summarized a flurry of qualityimprovement (QI) activities and pointedto new directions aimed at making theUS health care delivery system more ofa ldquolearning organizationrdquo15 The reportmay be summarized as articulating the

perspective that ldquoTechnology can fixwhatrsquos broken in health carerdquo16 Thisoptimism is consistent with urgentcalls for action that fall short of em-pirical guidance or even a clear path toimprovement One potential path tobring action together with learning canbe found in the form of applied socialscience17 As specifically applied to theenhancement of performance ldquoim-provement sciencerdquo has been de-scribed as the systematic effort to findldquoout how to improve andmake changesin the most effective way examiningthemethods and factors that best workto facilitate quality improvementrdquo14How to move QI forward is an ongoingsubject of discussion18ndash30

This article argues for the centrality ofscience including rigorous theory de-velopment and testing in moving thenationrsquos quality aims forward In partwe build off of work we presented atthe first 2 ldquoAdvancing the Science ofPediatric Quality Improvement Re-searchrdquo conferences in 2011 and201231ndash33 We identify gaps in the cur-rent theory and practice of QI researchand evaluation in health care and tosuggest approaches to closing thosegaps focusing on pediatric QI as ap-propriate We suggest specific designsshould be matched to specific circum-stances for considering health careimprovement and its evaluation Fi-nally we suggest how practical expe-rience can help to build a theory ofapplied QI in health care that can helpresearchers to identify generalizablefindings from QI evaluation and re-search and help the nation achieve itsgoals

QUALITY IMPROVEMENT

A large stream of QI activities in healthcare are derived from the philosophyof total quality management and thework of Edwards Deming and WalterShewhart34 Themodern era of QI in healthcare began with efforts to stimulate

providersrsquo use of clinical practiceguidelines or to help providers meetperformance goals benchmarkedagainst data from performance mea-sures35ndash37 This era began after it be-came clear that merely producingclinical practice guidelines and pas-sively disseminating them was notleading to change Although there aresome compelling examples of suc-cessful QI efforts focused on specificclinical problems at the clinical micro-system level1538 the current emphasisin QI is on broader systemic effortsdesigned to change the environment inwhich providers practice This empha-sis includes concepts such as systemsreengineering39 delivery system trans-formation 6 s LEAN and systemschange2326 We consider QI broadly toconsist of those systematic data-basedactivities that focus on changing theproduction function of health care withan aim toward improving outcomesandor efficiency Although the fullnessof QI may be manifest in iterative cyclesof measurement and improvement weagree with Berwick1840 that the aimsare more important than the purity ofthe term the empirical emergence ofevidence and understanding createsopportunities to iteratively posit andtest conceptual models and theoriesHis other point in 1996 that we need tobroaden our ideas of what constitutesgood science is as relevant today asthen

In 2003 Galvin and McGlynn41 consid-ered 3 social science models ChangeTheory Tipping Point and DiffusionTheory to explore why QI had notresulted in a fundamentally improvedhealth care system They recom-mended a strategy that used perfor-mance measurement to drive changeTen years later observers are stillasking why health care quality haslargely not improved34 and attemptingto develop frameworks to help guideimprovement42

SUPPLEMENT ARTICLE























Current Theory



SUPPLEMENT ARTICLE











CONCLUSIONS











SUPPLEMENT ARTICLE



















APPENDIX










REFERENCES

























SUPPLEMENT ARTICLE





































































SUPPLEMENT ARTICLE






References



































Current Theory



SUPPLEMENT ARTICLE











CONCLUSIONS











SUPPLEMENT ARTICLE



















APPENDIX










REFERENCES

























SUPPLEMENT ARTICLE





































































SUPPLEMENT ARTICLE






References
























Current Theory



SUPPLEMENT ARTICLE











CONCLUSIONS











SUPPLEMENT ARTICLE



















APPENDIX










REFERENCES

























SUPPLEMENT ARTICLE





































































SUPPLEMENT ARTICLE






References























CONCLUSIONS











SUPPLEMENT ARTICLE



















APPENDIX










REFERENCES

























SUPPLEMENT ARTICLE





































































SUPPLEMENT ARTICLE






References






















SUPPLEMENT ARTICLE



















APPENDIX










REFERENCES

























SUPPLEMENT ARTICLE





































































SUPPLEMENT ARTICLE






References































APPENDIX










REFERENCES

























SUPPLEMENT ARTICLE





































































SUPPLEMENT ARTICLE






References





















APPENDIX










REFERENCES

























SUPPLEMENT ARTICLE





































































SUPPLEMENT ARTICLE






References

















































































SUPPLEMENT ARTICLE






References











































SUPPLEMENT ARTICLE






References


















References




















Documents

Assessing Quality Improvement in Health Care: Theory for … · Assessing Quality Improvement in Health Care: Theory for Practice abstract OBJECTIVES: To review the role of theory