CLINICAL, SIGNIFICANCE AS IT RELATED TO EVIDENCE BASED�

PRACTICE; A CONCEPT ANALYSIS�

By�

SUSAN KRISTINE SCHMIDT BRlTNER�

A Non-Thesis Project submitted in partial fulfillment of� the requirements for the degree of�

MASTER OF NURSING�

WASHINGTON STATE UNIVERSITY� College of Nursing�

DECEMBER 2010�

To the Faculty of Washington State University:

The members of the Committee appointed to examine the dissertation/thesis of SUSAN

KRISTINE SCHMIDT BRUNER find it satisfactory and recommend that it be accepted.

Cindy Corbett, Ph.D., Chair

Brian Gates, Pharm.D.

~r~~~ Ryan P. Townsend, M.N.

For Submission to: Journal ofAdvanced Nursing

Abstract Aim. Analyzes the concept of clinical significance in relation to evidence-based practice. Background. There has been insufficient inquiry into the evidence base of the definition and use of the term clinical significance in nursing. Delineating a common conceptual meaning and definition for clinical significance will enhance its usefulness in research and practice. Methods Rodgers' evolutionary approach to concept analysis provides the framework for interpreting clinical significance in the context of its uses, measurements, and defInitions in the nursing literature. After applying inclusion and exclusion criteria and a sampling method, 53 articles were included in the analysis of clinical significance. Using Rodgers' rigorous, evolutionary method of concept analysis, the concept of clinical significance was examined for its significance, use and application as it has unfolded between 1999 and 2010. Results Findings indicate that the nursing literature contains various, disparate definitions and applications ofclinical significance. Conclusion The disparate uses of clinical significance negatively impact standardizing and quantifying research outcomes to discern evidence-based practices. The authors propose a definition of clinical significance inclusive of the multifarious uses that were revealed in the literature and conclude tllat there is a need for professional nursing consensus to define the term. A standard operational definition ofclinical significance would enable consistency as clinicians interpret research findings and facilitate translating research to practice.

Introduction

In 2002, the Mayo Foundation for Medical Education, in collaboration with over 30

collective authors as members of the Clinical Significance Consensus Meeting, published a

series of reports describing several problems with the use of the term clinical significance (CS).

These problems included inconsistent measurements, changing definitions over time, varying

opinions among stakeholders, theoretical inconsistencies, and unclear guidelines for determining

CS in group versus individual differences or single item versus multi-item instrument scores. The

consensus panel asserted that these problems interfere with the ability of clinicians to clearly

interpret research data and detract from the credibility ofCS as a concept (Guyatt et. aI., 2002;

Sloan et. al., 2002; Sloan et. al., 2003).

The importance of CS as concept was illustrated in its comparison to statistical

significance: while the P values of a statistically significant fmding indicate the likelihood of the

change being caused by chance, CS goes beyond statistical significance to identify whether the

statistically significant change or difference was large enough to have implications for patient

care (Sloan et. aI., 2002; Pittman & Rawls, 2008). The potential utility ofa concept that

consistently indicated significant patient change was perceived to have positive implications for

stakeholders including researchers, clinicians, society, and patients. Thus further research by

physician scientists ensued to resolve identified disparities and determine valid and reliable

measurements of CS. Substantive progress has been made to develop and define methods of

quantifying CS. The psychology/psychiatry (Jacobso~ Follette & Revenstorf, 1984; Jacobson &

Truax, 1991; Jacobson, Roberts, Berns & McGlinchey, 1999) and pain management (Mantha et.

al., 1993) specialties have operationalized CS and formal measurement methods. Oncology

(Frost et. aI., 2002; Guyatt et. al., 2002; Sloan et. aI., 2002; Sprangers et. aI., 2002 & Symonds,

Berzon, Marquis & Rummans, 2002), pulmonary medicine (Lacass et. al., 2002; Puhan et. al.,

2004), rehabilitation (Haley & Fragala-Pinkham, 2006), surgery (Copay et. al., 2008; Carreon,

Glassman, Campbell & Anderson, 2010), and other medical disciplines have made great strides

in this important undertaking. These combined medical efforts lend credence to the importance

ofundertaking efforts to determine valid and reliable measures for CS.

As an indicator ofpatient change, CS has evolved in healthcare to provide measures

observable to the clinician/researcher and patient. The patient's self assessment of change has

evolved as the central measure of CS. One current interpretation of CS as a research and clinical

term is as follows: a minimal clinically significant/important difference represents the smallest

difference in score in the outcome of interest that informed patients or proxies perceive as

important, either beneficial or harmful, and which would lead the patient or clinician to consider

a change in patient management (Schunemann et. aI., 2005). Inclusion of the patient into the

calculation of CS has important implications for evidence-based practice (EBP) because patient

values are critical to the achievement of best practices (Jones, 2010; Melnyk & Fineout­

Overholt, 2011).

While medicine has made substantial progress in defining CS, nursing has not made

similar attempts to define the term and substantiate it as a measure ofpatient response to

treatment that fulfills evidence-based standards. Indeed, there has been no formal, integrated

theoretical research undertaken by the nursing discipline to define an operational use of the term

CS (Jeans, 1992; Burns & Grove, 2009; Dysvik et aI, 2009) and there is a general lack of clarity

about CS in the nursing literature. It has been variously defined as anything of relevance to

nursing (Jeans, 1992), anything that is worth changing practice for (Fawcett, 2009), and- a value

judgment (LeFort, 1993). In the contemporary nursing literature, CS has been variously defmed

as: a measure of patient response to treatment based on pre-determined thresholds (Pittman &

Rawls 2008); a change that is at least twice the standard deviation of difference in scores due to

measurement errors (Dysvik, Kvaloy, Stokkeland, Natvig, 2009), and meaningful individual

change exceeding the Reliable Change Index value that results in the final (post-test) score

falling within the 'normal range' (Baird, Worral, Haslam & Haslam, 2010). Clearly, within the

nursing discipline a standard definition of CS has not materialized.

Standardizing defInitions, measures, and methodology is essential to disseminating best

practices. The lack of clarity when defIning CS contributes to uncertainty about the quality of

evidence represented by a CS change. According to Rodgers, a concept analysis is necessary

when there are inconsistencies in terminology, utilization, or lack of theoretical coherence

(Rodgers, 2000). The purpose of this paper is to use Rodgers' evolutionary concept analysis to

discern a "state of the art" consensus for CS. A "state of the art" consensus is intended to

advance the development of the term to promote consistency in interpreting research fmdings

and facilitate translating research into evidence.

Methods

An extensive review of the literature was conducted. Literature searches were performed

to identify articles pertinent to clinical significance. The electronic databases CINAHL, lSI Web

of Knowledge, PubMed, Cochrane Database of Systematic Reviews, and Google Scholar were

searched using the following keywords: clinical significance, clinical significance measures,

clinically significant, clinically meaningful, clinically important, and evidence basedpractice.

The search included papers published between the January 1999 and December 2010. Additional

searches were completed on several websites, such as those of the Agency for Healthcare

Research and Quality, The Joint Commission, and the Center for Medicare and Medicaid

/1

Services. In addition to computer searches, reference lists of articles on clinical significance

provided additional publications that were reviewed.

A search for previous clinical significance concept analyses provided no results.

Inclusion criteria for articles used for this concept analysis were that they were printed in

English, published in peer-reviewed journals, had a nurse as the first author, and were relevant to

the main topic of clinical significance. Approximately 83 articles were located and abstracts

read. A total of 56 articles were selected for detailed review and analyzed to obtain

understanding of the many facets of clinical significance. Of the 56 articles used for analysis,

three did not use clinical significance as a research term. The final number ofarticles evaluated

in this concept analysis was 53.

Clinical significance has been studied and discussed by a wide range of international

professionals. Nursing articles were co-authored by physicians, pharmacists, social workers, and

mathematician researchers living and working in the United States, Canada, England, Italy,

Germany, Denmark, Sweden, Finland, China, and Singapore. International interest is indicative

of the relevance of the concept ofCS.

Proposed Definition Based on Findings

Rodgers asserted that the identification of surrogate terms, (e.g. terms that are frequently

interchanged with the concept of interest), promotes elucidating the unique and defining aspects

of the concept of interest. In the nursing literature, surrogate terms used most often as

synonymous with CS were the terms clinical importance, clinical relevance, and clinically

meaningful. While these terms were interchanged with CS, it is notable that CS was used more

exclusively when quantification, numbers or measures were applied to explain the importance of

the fmding; this observation is further described in subsequent sections. The common themes, as

they occurred in the nursing literature, are presented as the core attributes of the concept.

Synthesizing literature themes to identify core attributes of a concept provides the basis of the

"consensus of use" definition. Rodgers asserted that the "consensus of use" definition is

equivalent to a "state of the science" overview of a concept, thus it provides a baseline upon

which to guide theory development.

In accordance with the Rodgers' methodology, the first finding to be discussed is a

"consensus of use" definition of the tenn as derived from a thematic analysis of the literature.

The definition proposed is broadly inclusive of the various uses of the tenn: CS is a patient

related difference that the clinician/researcher has judgedfor nursing knowledge to guide

nursing interventions. This definition is reflective of the literature where in most cases CS was

described as resulting from changes in the status of patients which was judged (either

subjectively or quantitatively) as relevant to nursing knowledge and interventions. It is important

to note that this definition is proposed as an a posteriori observation of the existing literature;

further efforts are required to define CS more distinctly and specifically for the nursing

discipline.

Attributes of Clinical Significance

Patient related

CS is associated with three patient related themes: research directly involving patients;

patient-related policies and protocols, and medical equipment and procedures. The majority of

studies explicitly involved patients in either an inpatient or outpatient setting. Outpatient settings

were represented by individuals in rehabilitation (Riegel et. al., 2006; Baird et. al., 2010), or

nursing homes (Dowling, Graf, Hubbard & Luxenberg, 2007; Fjelltun et. aI., 2008), and those

receiving post-surgical care (Wilson & Hagadottir, 2006) or psychotherapeutic treatment

(England, Tripp-Reimer & Rubenstein, 2005) in their homes or in outpatient clinics. Inpatient

settings were represented by individuals receiving treatment for cancer (Cella et. aI., 2003; Bauer

& Romvari, 2009; Swanson & Koch, 2010) and psychiatric conditions (Bowers et. al., 2006), as

well as obstetrical care (Simpson, Parsons, Greenwood, & Wade, 2001; Callahan & Hynan,

2002; Waltman, Brewer, Rogers & May, 2004), pre and post-surgical care (Kurlowicz, 2001;

Gallagher & McKinley, 2009; ) and intensive care (Happ, Tuite, Dobbin, DiVirgilio-Thomas, &

Kitutu, 2004).

A smaller portion of the studies included indirect patient care issues - specifically

reviews ofhealthcare literature (Schaffer, Yoon, & Zadezensky, 2009) and evaluation ofpatient

care-related policies (Doughty et. aI., 2006) and protocols (Harrington, 2005). The smallest

category of studies assessed the accuracy ofmedical equipment and procedures (Gilbert, Barton

& Counsell, 2002; Farnell, Maxwell, Tan, Rhones & Philips, 2005; Zengin & Enc, 2008).

Difference: Change

CS was often asserted as having occurred when a difference was detected. The most

common differences were those designated as change. The most common changes attributed to

CS were patient responses to treatments. The usual indication of this was the improvement or

deterioration of a medical or psychiatric condition. Psychiatric treatment responses included

reduction of depression, anxiety or distress (Kurlowicz, 2001; Simpson, Parsons, Greenwood &

Wade, 2001; Callahan, & Borja, 2008; Gallagher & McKinley, 2009) and inpatient violence

(Bowers, Brennan, Flood, Lipang & Oladapo, 2006; Dowling, Graf, Hubbard, & Luxenberg,

2007; Selbaek, Kirkevold, & Engedal, 2008). Responses to treatment often involved pain

reduction and changed biopsychosocial functioning (Cella et aI, 2002; Frost et aI, 2002; Barrett,

Brown, & Mundt, 2007; Baird, Worral, Haslam, & Haslam, 2008; Dysvik, Kvaloy, Stokkeland

& Natvig, 2009).

Difference: Comparative

The second most common differences that were designated as CS in the literature were

comparative differences. Comparative differences occurred when a different condition was

detected in an individual, group of individuals, or issues related to patient care. In most studies, a

CS difference equated with detection of an abnormal psychiatric (Gallagher & McKinley, 2009),

medical (Barrett, Brown, & Mundt, 2007), functional (Dysvik et. aI., 2010) or pain condition

(Kokki et. al., 2003). In other studies the difference was related to an outdated policy (Doughty

et. aI., 2006), protocol (Harrington, 2005; Ryan-Wenger, Neal, Jones, & Lowe, 2010) or

equipment function (Gilbert, Barton, & Counsell, 2002). Adverse outcomes were also cited as

CS (Jenkins & Lindsey, 2010).

Clinician/researcher

Common themes ofjudging CS were researcher opinion (Wendler, 2003; Ward-Smith,

Korphage & Hutto, 2008; Hogan, 2009; Ryu et. al., 2010), and researcher opinion accompanied

by parameters established by expert opinion that involved: a) consensus panels (Farnell et. aI.,

2005; Doughty et al., 2006): b) government-sponsored guidelines (Skelly, Carlson, Leeman,

Soward, & Bums, 2009): c) institution-based guidelines (Gilbert, Barton, & Counsell, 2002;

Zengin & Enc, 2008): or d) recommended thresholds or cut-off scores suggested within

instruments (Dowling, Graf, Hubbard, & Luxenberg, 2007; Selbaek, Kirkevold & Engedal,

2008). A less common theme involved integrating patient judgments with researchers' judgments

to ascertain CS (Cella et. aI., 2003; Baird, Worral, Haslam, & Haslam, 2010; Dysvik et aI, 2010).

Judging CS was based upon a diversity of criteria and rationale. Often judgment was

based on a measure; the most common measure was mathematical. Mathematical measures

included applying instrument-specific numeric cut-off scores (Callahan & Borja, 2003; Dowling,

Graf, Hubbard, & Luxenberg, 2007; Selbaek, Kirkevold & Engedal, 2008; Swanson & Koch,

2010) using pre-determined, a-priori judgment by the researchers (Farnell, Maxwell, Tan,

Rhodes, & Philips, 2005), using a-posteriori judgments for moderate to large differences

(Kurlowicz, 2001; Happ, Tuite, Dobbin, DiVirgilio-Thomas, & Kitu~ 2004; Bowers, Brennan,

Flood, Lipang, & Oladapo, 2006; Schaffer, Yoon & Zadezensky, 2009; Skelly, Carlson, Leeman,

Soward, & Burns 2009), and using a-posteriori judgment of small to moderate changes or

differences (Simpson, Parsons, Greenwood, & Wade, 2001; Wendler, 2003). The second theme

involved measuring patient feedback accompanied by clinician-rating functioning (Cella et aI,

2002; Frost et aI, 2002; Barrett, Brown, & Mundt, 2007; Baird, Worral, Haslam, & Haslam,

2008; Dysvik, Kvaloy, Stokkeland & Natvig, 2009) CS judgments were not based on measures.

Instead, methods of determining CS were attributed to important occurrences including

identifying important critical occmrences (Jacobs, Apatov, & Glei 2005; Jenkins & Lindsey,

2010), discovering a deficient protocol (Ruchala, Metheny, Essenpreis, & Borcherding, 2002;

Schaffer, Yoon & Zadezensky, 2009) or detecting an important patient condition (Harrington,

2005; Hogan, 2009).

Nursing Knowledge to Guide Interventions

Generally, findings were accompanied by recommendations for application to clinical

practice. The recommendations varied in their level of specificity from general admonishments

for nurses to become more involved in policy making to prescriptive advice with specific

recommendations for practical application. Examples of recommendations to guide nursing

interventions included recommendations for medication prescribing (Schaffer, Yoon, &�

Zadezensky, 2009) treating post-surgical pain (Kokki, Kankkunen- Pietila, & Vehvilainen­�

Julkunen, 2003; Wilson & Helgadottir, 2006), managing medication responses (Bauer, &�

Romvari, 2009; Marrs & Zubal, 2009), and interpreting or implementing protocols, policies or�

procedures (Kurlowicz, 2001; Gilbe~ Barton, & Counsell, 2002; Ruchala, Metheny, Essenpreis,�

& Borcherding, 2002; Farnell, Maxwell, Tan, Rhones, & Philips, 2005; Harrington, 2005;�

Doughty et al, 2006; Zengin & Enc, 2008; Hogan, 2009; Swanson & Koch, 2010).�

Antecedents�

The main antecedent for CS was that the research involved patient-related variables. The

broad objective of the studies was to provide relief of patient distress or deficits. Patient distress

or deficits included pain (Kokki, Kankkunen, Pietila, & Vehvilainen-Julkunen,2003; Wilson &

Helgadottir, 2006), cognitive emotional distress (Kurlowicz, 2001; Simpson, Parsons,

Greenwood, & Wade, 2001; Callahan & Borja, 2003; Wendler, 2003; Dowling, Graf, Hubbard,

& Luxenberg, 2007; Gallagher & McKinley, 2009), medical conditions (Skelly, Carlson,

Leeman, Soward, & Burns 2009; Ryu et. al., 2010), and quality of life (Frost et al, 2002;

Barrett, Brown, & Mundt, 2007; Baird, Worral, Haslam & Haslam, 2008). Topics that indirectly

affected patient distress or deficits involved assessing policies (Hogan, 2009; Schaffer, Yoon, &

Zadezensky, 2009), protocols (Harrington, 2005) or equipment (Zengin & Enc, 2008).

Consequences

The consequences of CS, based on the manner in which CS was applied in the literature,

include uncertainty about implementation of CS research fmdings. Because CS was often based

on non-standardized measures and definitions, the strength of the evidence is unclear. Strength of

evidence is defmed as the likelihood that a research finding will produce a measurable patient

outcome (Substance Abuse and Mental Health Services Association, 2006). As CS is currently

used in the research literature, it is not always associated with a measurable patient outcome.

Many of the practice recommendations were based on subjective researcher opinion as the basis

for determination of CS. Guidelines established for EBP place a low value on effecting nursing

practice changes that are based on opinion. (Melnyk & Fineout-Overholt, 2011). Additionally,

research that does not directly include patient feedback has come under scrutiny (Frost et. aI.,

2002; Lacass et. al., 2002; Puhan et. al., 2004; Haley & Frugala-Pinkham, 2006; Copay et. al.,

2008 & Carreon, Glassman, Campbell, & Anderson, 2010); placing further concerns about the

application of a CS finding to best practices. In recent years, lawsuits against nurse practitioners

have increased and one key recommendation to decrease risk included following evidence-based

practice guidelines (Edmunds & Scudder, 2009). In a worst-case-scenario, a practitioner who

does not understand or misinterprets CS in the literature could adjust nursing practice by

accepting as a priori the importance of a finding reported to be CS.

Related concepts

The term CS was often used interchangeably with related concepts to convey clinical

importance. These terms include "clinically relevant," "clinically important," and "clinically

meaningful." While CS was used interchangeably with these related concepts, it was

overwhelmingly used when quantification, numbers or measures were applied to substantiate a

research finding.

Discussion

The purpose of this concept analysis was to examine the predominant conceptual

meaning of CS using Rodgers' methodology. In an effort to bring a coherent understanding to

the concept, a definition of CS was developed based on common themes derived from its use

over the past ten years in the nursing literature. The end result was a proposed "state of the art"

understanding that is a synthesis of its myriad uses.

There are numerous problems associated with the use of CS in the nursing literature.

First, information from the analysis indicated a lack of consensus about the use of the term.

Thus, there is a multiplicity ofopinions, definition and uses of CS. The absence of a consistent

definition creates problems for clinicians trying to interpret validity, reliability, relevance, and

the overall safety of findings that are labeled CS. Second, because CS was commonly based on

researcher judgment, the term is often used subjectively and findings are prone to bias in favor of

positive results. Third, the majority of the studies did not incorporate the perspective of the

patient. These problems strongly suggested that a finding labeled as CS should not necessarily

infonn decisions about nursing practice. In addition, the fmdings from this concept analysis

inferred a need to establish a common definition of CS and general guidelines for its

measurement.

In attempting to delineate benchmarks for CS in nursing, it is important to consider some

of the problems encountered by researcher/clinicians in non-nursing disciplines. Some authors

asserted that while an optimal and highly desirable solution would be to develop a single rule,

policy, or guideline for assessing CS, such a goal was unrealistic (Ogles, Lunnen, & Bonesteel,

2001; Cella et aI., 2002; Guyatt et aI. 2002; Sloan et al., 2002). Other authors assert that a single

quantitative method for determining CS was overly simplistic (Jacobson, Roberts, Berns, &

McGlinchey, 1999) or suggested that as long as a common definition of CS was agreed upon, a

variety of different measures will produce results that are reasonably equivalent (Atkins, Bedics,

McGlinchey, & Beauchaine, 2005) Another challenge was that in order to make measures ofCS

meaningful, the perspectives patients, caregivers, clinician/researchers, and society or health

policy makers needs to be identified because judgment of CS differed among stakeholders (Frost

eta aI., 2002; Hajiro, & Nishimura, 2002; Greenstein, 2003; Pittman, & Rawl, 2008). Finally,

regardless of efforts to objectively quantify CS, subjective value judgments, while (ideally)

expertly informed, pervaded clinical management and research decisions (Guyatt et aI., 2002;

Hajiro, & Nishimura, 2002; Greenstein, 2003; van Roon et al., 2009).

To resolve some of these problems, it is becoming clear that the patient perspective

should be the foundation of CS (Kazdin, 1999; Frost et al., 2002; Guyatt et al., 2002; Guyatt et

aI., 2007; Metz et aI., 2007; Copay et aI., 2009). While actively soliciting patient perspectives has

not traditionally been important for [empiric] research tradition, there has been an increased

emphasis on incorporating patient feedback about treatment into research decisions whenever

possible (Dysvik et al., 2009; Molnar et al., 2009). Indeed there are current efforts to increase

awareness of patient perspectives in clinical and drug trials (Molnar et al., 2009). An emphasis

on the patient as the foundation of data about CS is not only appropriate to help inform

clinician/researcher decisions, but data derived from patient input is considered appropriate for

other stakeholders including society and policy makers. Finally, the inclusion of the patient

perspective into the judgment of CS is congruent with the holistic foundation of nursing practice

and with EBP standards. Indeed, patient input is receiving increased emphasis within the EBP

paradigm (Fawcett, 2010; Melnyk & Fineout-Overholt, 2011). Efforts to quantify patient

response and ascribe meaning to treatment have resulted in measures ofminimal and maximal

change, as well as measures ofpatient value judgments. These measures will be described next.

In delineating measures ofpatient response to treatment, there has been a need to identify

the minimal response to treatment. While there are a variety ofmethods to measure minimum

difference including minimum clinically important difference (MelD) and minimum clinically

important improvement (Mell), there are indications that the minimally clinically

significant/important difference (MID) will become the preferred standard to measure minimal

change (Schunemann et al., 2005; Barrett, Brown & Mundt, 2008). MID represents the smallest

difference in score in the domain of interest that patients perceive as important, either beneficial

or harmful, and which would lead the clinician to consider a change in the patient's management

(Guyatt et aI., 2007, p. 377). Variations of MID calculations may be modified to capture

intrapatient, interpatient or aggregate group differences (Cella et al., 2002). Additionally, select

statistical applications of MID, in conjunction with anchors derived from global and functional

scales or distributional methods can provide averaged MID values that are generalizable to other

studies (Copay et aI., 2008; Molnar et aI., 2009).

There are several ways to measure moderate to large or maximal change. Individual

differences are commonly measured with the Reliable Change Index, which indicates the

likelihood that the large magnitude of individual patient change is real and has not occurred by

chance (Jacobson, Follette & Revensdorf, 1984; Jacobson & Truax, 1991; Pittman & Rawl,

2008; Dysvik, Kvaloy, Stokkeland & Natvig, 2010). Measures of moderate to large group

treatment response are traditionally measured with effect size, a measure of the difference in

treatment effect between the treatment group and the comparator. However, this method has

come under scrutiny because it ignores the patient attribution of meaning (Cella et al., 2002;

Guyatt et al., 2002). Additionally, results expressed as effect size are not always generalizable to

other studies (Molnar et al., 2009). A number of measures are available, however, for

understanding the magnitude of effects that include patient perspective. Investigators may relate

changes in global score (e.g., quality of life score) to changes in scores in functional assessment,

clinical diagnoses, the patients' global ratings of the magnitude of change they have experienced,

or to the extent that patients rate themselves as feeling better or worse than other patients (Guyatt

et aI., 2007). Measures of maximum treatment response, while inclusive of pharmaceutical

treatment responses, often include responses to non-pharmaceutical treatments including

physical, psychological, rehabilitation, surgical, and multidisciplinary therapies (Cella et aI.,

2003; Dysvik et al., 2009; Metz et al., 2010).

There are other evolving aspects of CS that may be used to help the patient and clinician

make informed decisions about treatment plans that reflect patient values. Information about

likelihood of benefit to harm, lowest risk reduction, and highest benefit potential may be

measured in several ways; the most often cited are minimally clinically important difference

(MCID) and number needed to treat (NNT). MCID has been determined in various ways

including via anchor-based approaches, "between patients" score change, comparative

distributions, and social comparisons (Copay, Subach, Glassman, Polly, & Schuler, 2007). NNT

involves calculating the inverse of a percentage to determine the need to treat disorders caused or

exacerbated by a treatment effect. This is a percentage calculation that determines how many

patients need to receive a treatment in order to obtain one resulting outcome. Ifone of 100

individuals benefited from a treatment, then 100 individuals would have to be treated to obtain a

positive outcome, 200 for two, etc. (Melnyk & Fineout-Overholt, p. 91). These methods indicate

the plethora of evolving methods to quantify, predict, and assist patients to make informed

decisions about the CS of healthcare treatment options (Guyatt et al., 2007).

Implications for Further Research

While nursing has been relatively silent in regards to assigning a specific defmition and

measurement to CS, the medical community has been actively researching CS. In 2002, the CS

Consensus Group met to address problems and solutions related to CS. In 2007, they reconvened

and made suggestions for CS based on the ongoing medical trends and related research (Guyatt

et aI., 2007). The CS Consensus Group suggested that the definition of CS emphasize patient

response to a research or clinical treatment, and the importance and meaning the patient ascribes

to treatment effects (Guyatt et aI., 2007). Additionally, CS includes the risks and benefits of

alternative treatments and the likelihood of benefit or harm from those treatments (Guyatt et aI.,

2007). The CS Consensus Group suggested that measures to quantify CS are inclusive of the

methods described in the above section, and include the various measures of minimum and

maximum treatment effects, measures to indicate treatment outcome probabilities, and measures

ofpatient meaning ascribed to treatment (Guyatt et al., 2007).

In addition to emphasizing patient perspective, the CS Consensus Group suggested that

CS is most accurately appreciated by measuring multiple domains ofpatient experiences,

including cognitive, emotional, physical, functional, and social spheres of influence. This

conceptual model of CS assumes that patient response to treatment is experienced in multiples

aspects of their lives; thus, information of biological and physiological variables, symptoms

status, functional status; general health perceptions and overall quality of life provide the

evidentiary basis upon which informed determinations of CS are made. It is expected that

outcome measures specific to relevant medical or nursing research and/or clinical specialty are

included in the calculation of CS. Instruments to quantify the holistic measurement of CS from a

patient perspective continue to be developed; two examples of these instruments that can be

incorporated into the measurement ofCS include the Short Form-36 Health-Related Quality of

Life Questionnaire and the EuroQual-14. These measures are used widely in the US, Canada,

Australia, Japan and eight European countries (Hyak & Engle, 2010). While the patient

perspectives can make CS more holistic and meaningful, patient perspectives must also be

considered in relation to the research design. Ideally, patient perspectives to determine CS in

research will be conducted using a randomized clinical trial design to reduce the likelihood of the

Hawthorne effect.

There are compelling reasons for nursing to adapt the definition and suggested measures

recommended by the CS Consensus Group. For one, the measures are based on instruments that

have demonstrated validity and reliability. Second, the measures are quantifiable and replicable,

and are often the product of random controlled trials; thus the evidence produced is strong and

appropriate for application to best practice (Jones, 2010; Melnyk & Fineout-Overholt, 2011).

Third, the emphasis on patient perspective exemplifies the philosophical and theoretical

foundation upon which nursing is based. Finally, the patient perspective as measured in the

biopsychosocial domains reflects the core values of the holistic individual that are espoused by

nursing theorists.

The most immediate and tangible advantages to a common conceptual definition and

meaning of CS by nursing, regardless of which definition is selecte<L is less confusion and more

clarity. A consensus ofmeaning, a solid defmition and associated valid and reliable measures of

both quantitative and qualitative patient information may provide more fruitful and focused

research. From a clinician perspective, a solid defmition and explicit statistical nomenclature of

CS to address clinical applicability may result in improved interpretability of research findings.

User-friendly research findings may make research less burdensome for clinicians to read and

translate into best practices. Ultimately, the most enduring benefit of a common conceptual

definition and measurement for CS is the bridge it provides between research and practice, and

the facility with which it promotes the integration of research into evidence based practice.

Conclusion

Given the centrality of CS to interpreting research findings and applying them to practice,

there is a need to solidify the definition of and measurements for CS in nursing. National nursing

agencies including The National Institute for Nursing Research and Sigma Theta Tau

International should make standardizing CS a high priority for targeted funding. One method of

doing so would be to support a consensus convergence to review and select the optimal measures

of CS for nursing research. Research to increase knowledge about what constitutes measurement

and change or CS from the patient perspective is needed. Editors and peer reviewers should

encourage authors to include a discussion of CS. Discussions of CS should receive greater

emphasis in research journals. It is hoped that the preliminary fmdings from this concept

analysis presented in this article will facilitate the work of such a consensus forum.

References

Atkins, D. C., Bedics, J.D., McGlinchey, J. B. & Beauchaine, T. P. (2005). Assessing clinical

significance: Does it matter which method we use? Journal ofConsulting and Clinical

Psychology,73(5),982-989.

Baird, A., Worral, L., Haslam, C. & Haslam, R. (2008). Evaluation of a multi-disciplinary back

pain rehabilitation programme- individual and group perspectives. Quality ofLife

Research, 23(17), 357-366.

Barrett, B., Brown, R. & Mundt, M. (2008). Comparison of anchor-based and distributional

approaches in estimating important differences in common cold. Quality ofLife Research

20(17), 75-85.

Bauer, S. & Romvari, E. (2009). Treatment of chronic myeloid leukemia following Imatinib

resistance: a nursing guide to second-line treatment options. Clinical Journal ofOncology

Nursing, 1(5), 523-534.

Beaton, D. E., Boers, M. & Wells, G. A. (2002). Many faces of the minimal clinically important

Difference (MCID): a literature review and directions for future research. Current

Opinion in Rheumatology, 14, 109-114.

Bowers, L., Brennan, G., Flood, C., Lipang, M. & Oladapo, P. (2006). Preliminary outcomes ofa

trial to reduce conflict and containment on acute psychiatric wards: City Nurses. Journal

ofPsychiatric & Mental Health Nursing, 13(2), 165-172.

Brown, I. & Psaro~ A. (2007). Literature review ofnursing practice in managing obesity in

primary care: developments in the UK. Journal ofClinical Nursing 17, 17-28.

Burns, N. & Grove, S. K. (2007). Understanding nursing research: Building an evidence-based

practice (4th Ed.). St. Louis: Saunders Elsevier.

Callahan, J. L. & Hynan, M. T. (2002). Identifying mothers at risk for postnatal emotional

distress: further evidence for the validity of the perinatal posttraumatic stress disorder

questionnaire. Journal ofPerinatology, 22(6), 448-454.

Callahan, J. L. & Borja, S. E. (2008). Psychological outcomes and measurement ofmatemal

posttraumatic stress disorder during the perinatal period. Journal ofPerinatal and

Neonatal Nursing, 22 (1), 49-59.

Carreon, L.Y., Glassman, S.D., Campbell, M. J. & Anderson, P.A. (2010). Neck disability index,

short form-36 physical component summary, and pain scales for neck and arm pain: the

minimum clinically important difference and substantial clinical benefit after cervical

spine fusion. Spine Journal, 10(6), 469-74.

Cella, D. F. & Fridley B. (2003) Practical guidelines for assessing the clinical significance of

health-related quality of life changes within clinical trials. Drug Information Journal

37:23-31.

Cella, D.F., Zagari, M. J., Vandoros, C., Gagnon D. D., Huntz, H. J., Nortier J.W.R. (2003).

Epoetin alfa treatment results in clinically significant improvements in quality of life

(QOL) in anemic cancer patients when referenced to the general population Journal of

Clinical Oncology 21(2), 366-373.

Center for Substance Abuse Treatment (2006). Understanding evidence-basedpractices for co­

occurring disorders. (COCE Overview Paper 6). Rockville, MD: Author. Retrieved

October 11, 2010 from

http://coce.samhsa.gov/cod_resources/PDF/Evidence-BasedPractices(OP6). pdf

England, M., Tripp-Reimer, T. & Rubenstein, L. (2005). Exploration of the psychometric

properties of an inventory ofvoice experiences. Archives ofPsychiatric Nursing 19, 202­

217.

Farnell, S., Maxwell, L., Tan, S., Rhodes, A. & Philips, B. (2005). Temperature measurement:

comparison of non-invasive methods used in adult critical care. Journal ofClinical

Nursing, 14, 632-639.

Fawcett, J., & Garity, J. (2009). Evaluating research/or evidence-based nursingpractice.

Philadelphia: EA Davis Company.

Fjelltun, A.-M., Henriksen, N., Norberg, A., Gilje, F. & Normann, H.K. (2009). Functional

levels and nurse workload of elderly awaiting nursing home placement and nursing home

residents: a comparative study. Scandinavian Journal ofCaring Sciences, 23, 736-747.

Frost, M. H., Bonomi, A. E., Estwing - Ferrans, C., Wong, G.Y., Hays, R.D. & The Clinical

Significance Consensus Meeting Group (2002). Patient, clinician and population

perspectives on determining the clinical significance of quality-of-life scores. Mayo

Clinic Proceedings, 77, 488-4~94.

Gallagher, R. & McKinley, S. (2009). Anxiety, depression and perceived control in patients

having coronary artery bypass grafts. Journal ofAdvanced Nursing, 65(11), 2386-2396.

Gilbert, M., Barton, A. J. & Counsell, C. M. (2002). Comparison oforal and tympanic

temperatures in adult surgical patients (2002). Applied Nursing Research, 15(1),42-47.

Greenstein, G. (2003). Clinical versus statistical significance as they relate to the efficacy of

periodontal therapy. Journal ofthe American Dental Association, 134, 583-591.

Guyatt, G. H., Osoba, D., Wu, A. W., Wyrwich, K. W., Norman, G. R. & The Clinical

Significance Consensus Meeting (2002). Methods to explain the clinical significance of

health status measures. Mayo Clinic Proceedings, 77(4), 371-83.

Guyatt, G.H., Ferrans, C.E., Halyard, M.Y., Revicki, D.A., Symonds, T.L., Varricchio, C.G.,

Kotzeva, A., Valderas, J.M., Alonso, J.L., & The Clinical Significance Consensus

Meeting Group (2007). Exploration of the value ofhealth-related quality-of-life

information from clinical research and into clinical practice. Mayo Clinic Proceedings,

82(10), 1229-1239.

Hajiro, T. & Nishimura, K. (2002). Minimally clinically significant difference in health status:

the thorny path of health status measures? European Respiratory Journal, 19, 390-391.

Haley, S. M. & Fragala-Pinkham, M. A. (2006). Interpreting change scores of tests and measures

used in physical therapy. Physical Therapy, 86(5), 735-43.

Happ, M. B., Tuite, P., Dobbin, K., DiVirgilio-Thomas, D. & Kitutu, J. (2004). Communication

ability, method, and content among nonspeaking nonsurviving patients treated with

mechanical ventilation in the intensive care unit. American Journal ofCritical Care, 13,

(3), 210-218.

Harrington, L. (2005). Potassium protocols: in search of evidence. Clinical Nurse Specialist 19

(3), 137-141.

Hayat, M. J. (2010). Understanding Statistical Significance. Nursing Research, 59 (3), 219-223.

Hays, R. D., Woolley, J. M. (2000). The concept of clinically meaningful difference in health­

related quality-of-life research. How meaningful is it? Pharmacoeconomics, 419-423.

Hogan, R. (2009). Implementation of an oral care protocol and its effectiveness on oral

mucositis. Journal ofPediatric Oncology Nursing, 26, 125-134.

Jacobson, N. S., Follette, W.C., and Revenstorf, D. (1984). Psychotherapy outcome research:

Methods for reporting variability and evaluating clinical significance. Behavior Therapy,

15(4), 336-352.

Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining

meaningful change in psychotherapy research. Journal ofConsulting and Clinical

Psychology, 59 (1), 12-19.

Jacobson, N.S., Roberts, L.J., Berns, S.B. & McGlinchey, J.B. (1999). Methods for defIning and

determining the clinical significance of treatment effects: description, application and

alternatives. Journal ofConsulting and Clinical Psychology, 67(3), 300-307

Jacobs, J. L., Apatov, N., Glei, M. (2007). Increasing vigilance on the medical/surgical floor to

improve patient safety. Journal ofA"tivanced Nursing 57(5), 472-481,

Jeans, M. (1992). Clinical significance of research: a growing concern. Canadian Journal of

Nursing Research, 24(1), 1-2.

Jenkins, S. D., Lindsey, P. L. (2010). Clinical nurse specialists as leaders in rapid response.

Clinical Nurse Specialist 24(1), 24-30.

Jones, K. R. (2010). Rating the level, quality, and strength of the research evidence. Journal of

Nursing Care Quality, 25, 304-312.

Kazdin, A. E. (1999). The meanings and measurement of clinical significance. Journal of

Consulting and Clinical Psychology, 67(3), 332-339.

Kvien, T. K., Heidberg, T. & Hagen, K. (2007). Minimally clinically important

improvement/difference (MCII/MCID) and patient acceptable symptom state (PASS):

What do these concepts mean? Annals ofthe Rheumatic Diseases, 66, 40-41.

Kokki, A., Kankkunen, P., Pietila A. M., & Vehvilainen-Julkunen, K. (2003). Validation of the

parents' postoperative pain measure in Finnish children aged 1-6 years. Scandinavian

Journal ofCaring Sciences, 17, 12-18.

Kurlowicz, L. H. (2001). Benefits of psychiatric consultation-liaison nurse interventions for

older hospitalized patients and their nurses. Archives ofPsychiatric Nursing 15 (2), 53­

61.

Lacasse, Y., Brosseau, L., Milne, S., Martin, S., Wong, E., Guyatt, G. H. & Goldstein, R. S.

(2002). Pulmonary rehabilitation for chronic obstructive pulmonary disease. Cochrane

Database Systematic Review, 3: CD003793.

Lee, S. Y. & Kimble, L. P. (2009). Impaired sleep and well-being in mothers with low-birth­

weight infants. Journal ofObstetric and Gynecological Nursing, 38, 676-685.

LeFort, S. M. (1993). The statistical versus clinical significance debate. Image - The Journal of

Nursing Scholarship, 25(1), 57-62.

Mantha, S., Thisted, R., Foss, J., Ellis, J. E. & Roizen, M. F. (1993). A proposal to use

confidence intervals for visual analog scale data for pain measurement to determine

clinical significance. Anesthesia Analg., 77(5):1041-7.

Marrs, J., Zubal, B. A. (2009). Oncolocy nursing in a new era: optimizing treatment with

bevacizumab. Clinical Journal ofOncology Nursing, 13(5),564-572.

Melnyk, B. M. & Fineout-Overholt, E. (2005) Evidence-based practice in nursing & healthcare:

A guide to best practice. Philadelphia: Williams & Wilkins.

Metz, S.M., Wyrwich, K.W., Babu, A.N., Kroenke, K., Tierney, W.M. & Wolinsky, F.D. (2006).

Validity ofpatient-reported health-related quality of life global ratings of change using

structural equation modeling. Quality ofLife Research, 16: 1193-1202.

Molnar, F. J., Man-Son-Hing, M. & Fergusson, D. (2009). Systematic review of measures of

clinical significance employed in randomized controlled trials of drugs for dementia.

Journal ofAmerican Geriatric Society, (57), 3: 536-46.

Nisbet, A. T. & Mooney-Cotter, F. (2009). Comparison of selected sedation scales for reporting

opioid-induced sedation assessment. Pain Management Nursing, 10 (3): 154-164.

Ogles, B. M., Lunnen, K. M., and Bonesteel, K. (2001). Clinical significance: history,

application, and current practice. Clinical Psychology Review, 21(3): 421-446.

Pittman, J. & Rawl, S. M. (2008). Statistical and clinical significance are they one and the same?

Journal ofWound Ostomy Continence Nursing, 35(4): 374-376.

Puhan, M. A., Behnke, M., Devereaux, P. J., Montori, V. M., Braendi, 0., Frey, M. &

Schunemann, H. M. (2004). Measurement of agreement on health-related quality of life

changes in response to respiratory rehabilitation by patients and physicians - a

prospective study. Respiratory Medicine, 98(12): 1195-202.

Rodgers, B. L. (1989). Concepts, analysis and the development of nursing knowledge: the

evolutionary cycle. Journal ofAdvanced Nursing, 14 (4) 330-336.

Rodgers, B. L. & Cowles, K. V. (1997). A conceptual foundation for human suffering in nursing

care and research. Journal ofAdvanced Nursing, 25, 1048-1053.

Rodgers, B. L. (2000). Concept Analysis: An Evolutionary View. In B. L. Rodgers & K. A.

Knafl (Eds.). Concept Development in Nursing (2nd Ed.). Philadelphia: Saunders.

Ruchala, P. L., Metheny, N., Essenpreis, H. & Borcherding, K. (2002). Current practice in

oxytocin dilution and fluid administration for induction of labor. Journal ofGYN

Nursing, 31, 545-550.

Ryu, E., Kim, K., Cho, M.S., Kwon, I.G., Kim, H.S. & Fu, M. (2010). Symptom clusters and

quality oflife in Korean patients with hepatocelluar carcinoma. Cancer Nursing 33(1): 3­

10.

Schaffer, S.D., Yoon, S. & Zadezensky, I. (2009). A review of smoking cessation: potentially

risky effects on prescribed medications. Journal ofClinical Nursing, 18, 1533-1540.

Schunemann, H. M., Puhan, R., Goldstein, R., Jaeschke, R. J. & Guyatt, G. H. (2005).

Measurement properties and interpretability of the chronic respiratory disease

questionnaire (CRQ). Journal ofCOPD, 2 (1),81-89.

Selbaek, G., Kirkevold, o. & Engedal, K. (2008). Psychiatric and behavioural symptoms and the

use of psychotropic medication in special care units and regular units in Norwegian

nursing homes. Scandinavian Journal ofCaring Science, 22,568-573.

Shih-Yu, L. & Kimble, L. P. (2009). Impaired sleep and well-being in mothers with low-birth­

weight infants. Journal ofObstetric, Gynecologic & Neonatal Nursing 38(6),676-685.

Simpson, M., Parsons, M., Greenwood, J. & Wade, K. (2001). Raspberry leaf in pregnancy: its

Copay, A. G., Glassman, S, D., Polly, D.W. & Schuler, T. C. (2007). Understanding the

minimum clinically important difference: a review of concepts and methods.

Spine Journal, 7, 541-546.

Copay, A. G., Glassman, S. D., Subach, B. R., Berven, S., Schuler, T. C. & Carreon, L. Y.

(2008). Minimum clinically important difference in lumbar spine surgery patients: a

choice ofmethods using the Oswestry Disability Index, Medical Outcomes Study

Questionnaire Short Form 36, and pain scales. Spine Journal, 8(6),968-74.

DiCenso, A. (2001). Clinically useful measures of the effects of treatment. Evidence-Based

Nursing, 4, 36-39.

Doughty, D., Ramundo, J., Bonham, P., Beitz, J., Erwin-Toth, P., Anderson, R. & Rolstad B.S.

Sloan, J. A., Cella, D., Frost, M. H., Guyatt, G. H., Sprangers, M. A. G., Symonds, T. & The

Clinical Significance Consensus Meeting Group (2002). Assessing the clinical

significance in measuring oncology patient quality of life: introduction to the symposium,

content overview, and definition of terms. Mayo Clinic Proceedings, 77: 367-370.

Sloan, J. A., Vargas-Chanes, D., Kamath, C. C., Sargent, D. I., Novotny, P. .I., Atherton, P.,

Allmer, C., Fridley, B. L., Frost M. H. & Loprinzi, C.L. (2003) Detecting worms, ducks

and elephants: A simple approach for defining clinically relevant effects in quality-of-life

measures. Journal ofCancer Integrative Medicine 1(1):41-47.

Sprangers, M. A., Moinpour, C. M., Moynihan, T. I., Patrick, D. L., Revicki, D. A. & The

Clinical Significance Consensus Meeting Group (2002). Assessing meaningful change in

quality of life over time: a users' guide for clinicians. Mayo Clinic Proceedings, 77

(6):561-71.

Substance Abuse and Mental Health Services Administration, and Center for Mental Health

Services (2006). Understanding evidence-basedpractices for co-occurring disorders.

(COCE Overview Paper 6). Rockville, MD: Author. Retrieved September 29,2010 from

http://coce.samhsa.gov/cod_resources/PDF/Evidence-BasedPractices (OP6).pdf

Symonds, T., Berzon, R., Marquis, P., Rummans, T. A. & The Clinical Significance Consensus

Meeting Group (2002). The clinical significance ofquality-of-life results: practical

considerations for specific audiences. Mayo Clinic Proceedings, 77(6): 572-580.

Swanson, J. & Koch, L. (2010). The role of the oncology nurse navigator in distress management

ofadult inpatients with cancer: a retrospective study. Oncology Nursing Forum, 37(1):

69-76.

Tubach, F., Ravaud, P., Beaton, D., Boers, M., BOIrlbardier, C., Felson, D. T., Van der Heijde,

D., Wells, G. & Dougados, M. (2007). Minimal clinically important ilnprovement and

patient acceptable symptom state for subjective outcome measures in rheumatic

disorders. Journal ofRheumatology, 34, (5), 1188-1193

Van Roon, E. N., van den Bernt, P. M., Jansen, T. L., Houtman, N. M., van de Laar, M. A. &

Brouwers, J. R. (2009). An evidence-based assessment of the clinical significance of

drug-drug interactions between disease-modifying antirheumatic drugs and non­

antirheumatic drugs according to rheunatologists and pharmacists. Clinical Therapetuics,

31(8): 1734-1746.

Waltman, P. A., Brewer, J. M., Rogers, B. P. & May, W. L. (2004). Building evidence for

practice: a pilot study of newborn bulb suctioning at birth. Journal ofMidwifery &

Women's Health 49(1): 32-38.

Ward-Smith, P., Korphage, R. M. & Hutto, C. J. (2008). Cost comparison between two pediatric

palliative care programs. Nursing Economics, 26(3), 2008.

Wendler, M.C. (2003). Effects of Tellington touch in healthy adults awaiting venipuncture.

Research in Nursing & Health, 26, 40-52.

Wilson, M. E. & Helgadottir, H. L. (2006). Patterns of pain and analgesic use in 3-to-7-year-old

children after tonsillectomy. Pain Management Nursing, 7(4): 159-166.

Zengin, N. & Enc, N. (2007). Comparison of two blood sampling methods in anticoagulation

therapy: venipuncture and peripheral venous catheter. Journal ofClinical Nursing, 17(3):

386-393.