From Concept to Implementation: The Challenges Facing Evidence-Based Social Work

  • Published on

  • View

  • Download

Embed Size (px)


From Concept to Implementation: The Challenges Facing Evidence-Based Social Work. Faculty Research & Insights: A Series Featuring CUSSW Faculty Research Edward Mullen & Aron Shlonsky September 22, 2004. Evidence-Based Social Work Is A Concept Awaiting Implementation. Much discussed - PowerPoint PPT Presentation


<ul><li><p>From Concept to Implementation: The Challenges Facing Evidence-Based Social WorkFaculty Research &amp; Insights: A Series Featuring CUSSW Faculty Research</p><p>Edward Mullen &amp; Aron ShlonskySeptember 22, 2004</p></li><li><p>Evidence-Based Social Work Is A Concept Awaiting ImplementationMuch discussed Required in EPAS Not clearly Defined Not yet implemented in pure form</p><p>--- the conscientious, explicit and judicious use of current best evidence in making decisions regarding the welfare of service-users and carers (Sheldon, 2003, p. 1).Placing the clients benefits first, evidence-based practitioners adopt a process of lifelong learning that involves continually posing specific questions of direct practical importance to clients, searching objectively and efficiently for the current best evidence relative to each question, and taking appropriate action guided by evidence (Gibbs et al., 2003, p. 6).Evidence-based medicine has been defined as --- the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients (Sackett, et al., 1996, p. 71); and the "integration of best research evidence with clinical expertise and patient values" (Sackett, Straus, Richardson, Rosenberg, &amp; Haynes, 2000, p. 1).</p></li><li><p>Origin of Evidence-Based Social WorkAdaptation of evidence-based approaches found in medicine and healthcareEvolution from earlier attempts to join policy-practice-researchEffectiveness researchEclecticismEmpiricism &amp; the scientist-practitioner modelSocial intervention researchOutcomes measurementSystematic review methodologies meta-analysisEileen Gambrill early translator (1999)Leonard Gibbs first workbook-text (2003)Aaron Rosen &amp; Enola Proctor on guidelines (2003)</p></li><li><p>The EBP ModelSackett et al., 1997</p></li><li><p>Newest EBP ModelHaynes, Devereaux, and Guyatt, 2002</p></li><li><p>Steps of EBPBecome Motivated to Apply EBPStep 1Convert information need (prevention, assessment, treatment, risk) into an answerable question.Step 2Track down current best evidence.Step 3Critically appraise the evidence.Step 4Integrate critical appraisal with practice experience, clients strengths, values, and circumstances.Step 5Evaluate effectiveness and efficiency in exercising steps 1-4 and seek ways to improve them next time.Step 6Teach others to follow the same process</p><p>Gibbs, 2003</p></li><li><p>The Cycle of EBPActuarial risk assessmentOther valid assessment measures?Effective servicesAppropriate for this client?Barriers (e.g., cultural conflictClients preferred course or at least willing to try?Adapated from Haynes, Devereaux, and Guyatt, 2002; as presented in Shlonsky and Wagner, in press.</p></li><li><p>Hallmarks of EBPIndividualized assessment (strengths, needs, values, preferences of client)Well-formulated question (database query)Well-executed search of research literatureDeciding its applicability to clients Considering the evidence together with the values and preferences of the client.Sackett et al., 1997; Gibbs, 2003</p></li><li><p>Origins of Evidence-Based Healthcare &amp; Public PolicyUsed to inform health care policy at hospital, regional, &amp; national levels (Davies &amp; Boruch, 2001; Romanow, 2002; Gray, 2001; Davies, Nutley, &amp; Smith, 2000)Became component of New Labours modernizing government strategy - extended to other sectors including social care (Davies, 2004; Walker, 2001)Facilitated by:Cochrane &amp; Campbell CollaborationsUK ESRCs evidence-based practice &amp; policy centresUK Social Care Institute for ExcellenceSweden Institute for Evidence-Based Social WorkPublications e.g., Evidence-based Policy: A Journal of Research, Debate and Practice</p></li><li><p>Policy ApplicationsAn approach to policy that:helps people make well informed decisions about policies, programs &amp; projects EBPolicy contrasts with opinion-based policywhich relies heavily on either the selective use of evidence (single studies irrespective of quality)untested views of individuals or groups</p><p>Philip Davies (2004), Director of Policy Evaluation in the Cabinet Office of the UK Prime Ministers Strategy Unit </p></li><li><p>What is evidence-based healthcare?Discipline centered on evidence-based decision-making about:-Groups of patients -PopulationsEvidence from wide range of disciplinesUse of scientific research findings and logicHealthcare problemsHealth improvementDecision-making in healthcare</p><p>Gray 2001</p></li><li><p>Evidence-Based Policy DefinitionDecision-making process in which policy-makers, purchasers, managers, or practitioners together with key stakeholders make decisions about problem or need assessment, goals &amp; objectives, interventions &amp; outcomes</p></li><li><p>EBPolicy Process (Gray, 2001)</p></li><li><p>Sources of Evidence in EBP</p></li><li><p>EBPolicy StepsFinding and appraising evidence pertaining to the decision at hand: Assessment or intervention optionsIntended outcomesResearch methodDeveloping organizational capacityGetting the evidence into practicePreparing a policyMaking cultural changesDesigning systems for implementationImplementationMonitoringAudit</p></li><li><p>Examples of Evidence-based Policy InitiativesEVIDENCE AND POLICY: A JOURNAL OF RESEARCH, DEBATE AND PRACTICEESRC Evidence Network - UK Centre for Evidence Based Policy &amp; PracticeU.K. Cabinet OfficeUniversity of St. AndrewsRomanow Commission CanadaU.S. Implementing Evidence-based Practices Project</p></li><li><p>Arguments For Evidence-Based Social Work (EBSW)Enhances decision qualityFosters learning of assessment skillsIncorporates client values and expectationsFosters evidence search and appraisal skillsMakes best use of best evidenceFramework for self-directed, life-long learningIdentifies gaps in knowledgeCommon interdisciplinary languageSackett et al, 2000</p></li><li><p>Arguments For Evidence-Based Social Work (EBSW)Eschews authoritarian practices and policiesPromotes SW ethics through:Informed consentUsing effective servicesWisely allocating scarce resourcesInvolving clients in practice processIndividualizing &amp; personalizing decisionsGambrill, 2003</p></li><li><p>Arguments For Evidence-Based Social Work (EBSW)Helping clients develop critical appraisal skillsInvolving clients in design and critique of practice and policy related researchInvolving clients as informed participants who share in decision makingRecognizing clients unique knowledge in terms of application concernsPromoting transparency and honestyEncouraging a systemic approach for integrating practical, ethical and evidentiary issuesMaximizing the flow of knowledge and information about knowledge gapsGambrill, 2003</p></li><li><p>Challenges Facing EBSWEvidence-of-effectivenessAuthorityConflicting hierarchyDefinition of evidenceShortage of evidenceVariation by field of practiceInflation of evidenceSmall &amp; transient effectsAssessment validityNomothetic versus ideographic knowledgeEfficacy versus effectivenessDiffusion, adoption and implementationPolicy &amp; system level issuesTraining &amp; sustaining</p></li><li><p>Evidence-of-effectiveness ChallengeA key assumption of EBP is that it results in better outcomes than other forms of practiceYet, this assumption is not empirically testable:This proof is no more achievable for the new paradigm (EBM) than it is for the old, for no long-term randomized trials of traditional and evidence-based medicine are likely to be carried out. (Evidence-Based Medicine Working Group, 1992, 2424)</p></li><li><p>Authority ChallengeAs originally conceived the founders of evidence-based medicine envisioned that medical practitioners would seek out evidence &amp; appraise the quality of that evidence, alone or in teams of practitionersThe new paradigm puts a much lower value on authority. The underlying belief is that physicians can gain the skills to make independent assessment of evidence and thus evaluate the credibility of opinions being offered by experts (Evidence-Based Medicine Working Group, 1992)</p></li><li><p>Authority ChallengeLack of evidence that EBM can be transmitted into practiceMedical students have not shown an interest in becoming evidence-based practitioners only evidence usersAs evidence users rather than independent appraisers, practitioners are simply replacing one authority with another researchers are the new authorities!</p></li><li><p>Authority ChallengeTo date it has not been shown that average practitioners can gain the skills needed to conduct independent assessments of the evidencePractitioners can become evidence-users not evidence-based practitioners say Upshur &amp; TracyEvidence-users accept evidence that has been vetted by others thus replacing one authority for another! But what is the basis for the authority of the evidence appraisers?</p></li><li><p>Conflicting Hierarchy ChallengeHierarchy of Evidence Ranked by QualityN of 1 randomized trialsSystematic reviews/Meta-analysis of randomized trialsRandomized Controlled TrialsSystematic review of observational studies addressing patient-important outcomesPhysiological/Laboratory experimentsUnsystematic clinical observation</p></li><li><p>Conflicting Hierarchy ChallengePractice recommendations (e.g., practice guidelines) are to be based on the soundness of the evidenceMany evidence hierarchies have been published &amp; these hierarchies are not commensurable (e.g., randomized, controlled trials and meta-analysis versus well designed epidemiological studies)Fundamentally these hierarchies do not rest on evidence but rather on the consensus of experts the beliefs of the few!</p></li><li><p>Definition of Evidence ChallengeBecause contemporary health care and human services are multidisciplinary there are many differing perspectives that require considerationAccordingly, the concept of a univocal understanding of evidence is not sustainableA restrictive view of evidence is not sustainable</p></li><li><p>Shortage of Evidence ChallengeThe question that faces proponents of EBP is whether there are enough high-quality studies so that evidence-based decisions can be made.Surprisingly for a field that places a high premium on research, few studies have examined this.The conclusion at this point, based on just a few studies, is that there are still many decisions that are made that are not based on good evidence, but the picture is not nearly as bleak as opponents to EBP would have us believe. Professionals must remember, though, that when they make decisions for which little or no evidence exists, that they should exercise caution and perhaps be even more vigilant in monitoring outcomes.</p></li><li><p>Shortage of EvidenceVaries by field of practiceConfounded by inflation of evidence by proponents and usersClouded by dominance of small and transient effectsApplies to assessment evidence as well as intervention evidence</p></li><li><p>Shortage of Evidence Varies by Field of PracticeAdequacy varies considerably by field of practiceMost robust is mental healthIn other fields of practice much more of a patchworkUnevenness of the research base across fields of practice looms as a major challenge</p></li><li><p>Inflation of Evidence ChallengeBias in reported research due to:Tendency for investigators or assistants to give a leg up to interventions they favour - expectation for improvementDependence on client self-report to measure intervention outcomes which may reflect social desirability, expectancy, &amp; cognitive dissonance effects</p></li><li><p>Small &amp; Transient Effect Size ChallengeThe rule is small effect sizes which may not be clinically or social importantGains frequently do not persist, especially for complex and chronic problems such as addressed by social workers</p></li><li><p>Assessment ChallengeReliable, valid, &amp; relevant assessment tools are needed to support evidence-based social work practiceRapid assessment measures have contributed greatly (Corcoran &amp; Fisher, 2000)Social work practitioners generally do not use any standardized assessment procedures to guide their practice (Mullen &amp; Bacon, 2004)</p></li><li><p>Shortage of Evidence ChallengeEvidence-based social workers must remember that when they make decisions for which little or no evidence exists, that they should exercise caution and perhaps be even more vigilant in monitoring outcomesKnowledge gaps point the way to needed researchMethodological corrections should be instituted to deal with the inflation of evidence issueWhere interventions are found to have small with transient effects of limited importance this should be acknowledged and addressedThe underutilization of assessment instruments should be corrected by enhance training in the use of appropriate instruments &amp; dissemination of information about available instruments.</p></li><li><p>Challenge of Applying Results to IndividualsResults of RCTs are analyzed by comparing the mean score of the experimental group against that of the placebo or control group (or some comparable summary statistic). This masks the fact that there is always individual variability around the means, &amp; overlap in the distributions of scores for the two groups. The result of this is that a proportion of people in the experimental group actually do worse than some in the control group and, conversely, some in the comparison group improve more than some people in the active treatment group.Practitioners cannot blindly apply a proven procedure and assume that a particular individual receiving that procedure will benefit</p></li><li><p>Challenge of Applying Results to IndividualsWe are at least able to quantify the probability with which an individual person will respond to a given procedure (Number Needed to Treat)Alternative to using evidence-based interventions with their known rate of failure is to use unproven procedures, based only on the hope that they may workPractitioners can and should view each case or situation as an N = 1 study and collect data</p></li><li><p>Efficacy Versus Effectiveness ChallengeThe efficacy versus effectiveness challenge requires a careful consideration of the trade off between internal and external validityClaims to evidence-based practice typically must be grounded in random, controlled trials which usually translate into efficacy studiesSuch studies do not address how effective such interventions would be in real world contextsAt the present time there is a paucity of evidence of relevance to social work based on effectiveness studies conducted in real world contextsit may be more efficient to begin with studies based on conditions as realistic as possible so as to avoid the misinformation that too frequently comes from misinterpretations of efficacy studies</p></li><li><p>Adoption &amp; Implementation ChallengeTwo major challenges:(1) adoption &amp; effective implementation of evidence-based practice in social agencies(2) educating for evidence-based practiceSisyphus</p></li><li><p>Training, Time, Resources ChallengesEBP requires:Training in search techniquesTraining in critical appraisalComputer resourcesElectronic resources</p></li><li><p>Killer BsLow BASE rateIncompatible client or community BELIEFSBAD BARGAIN in terms of scarce resourcesBARRIERS too high</p></li><li><p>Adoption &amp; Implementation ChallengeMotivate core groupAcceptance of EBPFacilitate adoptionImplementation of effective servicesEvaluate outcomes</p></li><li><p>Gira, Kessler and Poertner (2004)Survey of reviews of adoption of research evidence medical practiceEducational outreach visits and audit and feedback showed weak to moderate effectsCertain types of continuing education and the use of computers as decision aids showed moderate effectsThere are no magic bulle...</p></li></ul>


View more >