From evidence to policy, programmes and · Chapter 22: From Evidence to Policy 271. drawnom fr a number of fields including evidence-based medicine, epidemiology, human development

Chapter 22: From Evidence to Policy

269

Chapter 22

From evidence to policy, programmes and interventions

David FergussonChristchurch Health and Development Study, University of Otago, Christchurch

Stuart McNaughtonWoolf Fisher Research Centre, The University of Auckland

Harlene HayneDepartment of Psychology, University of Otago

Chris CunninghamResearch Centre for Māori Health & Development, Massey University, Wellington

Summary• This chapter considers the issues that arise in the translation of research-based

evidenceintoeffectiveapplications:policies,programmesandinterventions.

• Three types of application are identified: universal applications targeted at alladolescents; prevention programmes targeted towards ‘at risk’ young people andtheirfamilies;andinterventionsthatfocusontreatingyoungpeoplewithproblematicoutcomes.

• ThechapterthenoutlinesthekeyelementsofthePreventionScienceapproachincluding:(a)theuseofmeta-analysisandsystematicreviewtoidentifyeffectiveprogrammes;(b)theimportanceofadequatepilottestingandprogrammedevelopment;(c)theuseofrandomisedtrialsandothermethodstoassessprogrammeefficacy;and(d)issuesintakingprogrammestoscale.

• Considerationisalsogiventotheissuesthatariseintheimplementationofprogrammesincluding the importance of staff training, organisational factors, client factors, andculturalfactorsinmaintainingprogrammefidelityandquality.

• The Prevention Science approach described above is compared with the KaupapaMāorimodelandthedifferencesbetweentheseapproachesexplained.Amodel(HeAwaWhiria–BraidedRivers)forreconcilingthesedifferentapproachesisdescribed.


270

• Considerationisgiventothetrainingandworkforcerequirementsthatareneededtoensuregreaterutilisationofsystematic(WesternScience;KaupapaMāori)approachestopolicydevelopment,implementationandevaluation.

1. IntroductionTheprecedingchaptersprovidearichmenuofrecommendationsaboutthewaysinwhichapplication policies, programmes and interventionsmay be developed to improve theoutcomes and circumstances of adolescents inNewZealand. Broadly speaking, theserecommendationsrelatetothreetypesofapplication[1]:

Universalpoliciesthatareappliedtoalladolescents.AnexampleofthistypeofpolicyisillustratedbytherecommendationinChapter20thatconsiderationshouldbegiventoraisingthelegaldrinkingage.

Programmestargetedtowards‘atrisk’youngpeoplesortheirfamilies.Theseprogrammesarepredominantlypreventionprogrammesthatseektomitigatetherisksforyoungpeoplewhoareexposedtosocial,economic,educational,familyorrelatedchallenges.ExamplesofthisapproachareprovidedbythehomevisitingandcentrebasedprogrammesreviewedinChapters3and5.

Interventionsthataretargetedtowardsyoungpeoplewhohaveexperiencedproblematicoutcomes.Examplesoftheseprogrammesincludethecognitivebehaviouraltreatmentsfordepression,alcoholabuseanddrugabusereviewedinChapters16,19and20.

Whilethepresentreportreviewsandrecommendsawiderangeofpolicyoptions,thecriticalreaderisentitledtoasktwogeneralquestionsabouttheseproposals:

Thefirstquestionis,“Howcanwebesurethatthepolicies,programmesandinterventionsbeingrecommendedare,infact,effectiveinaddressingwhattheypurporttoaddress?”.

Thesecondandperhapsmore importantquestion is,“Will thesepolicies,programmesandinterventionsworkinNewZealandand,ifso,whatstepsarerequiredtoensuretheirsuccessfulimplementation?”.

Thefocusofthepresentchapterisonaddressingbothoftheseissues.Specifically,thechapterreviews:

• The use of the Prevention Science Paradigm as a framework for identifying,implementingandevaluatingpolicies,programmesandintervention;

• KaupapaMāoriperspectivesonpolicydevelopment,andprogrammeselectionandevaluation;and

• ReconciliationofWesternScienceandKaupapaMāoriperspectivesusing theHeAwaWhiria(BraidedRivers)modelproposedbyMacfarlane.

2. The Prevention Science approachWith the rapidly growing body of knowledge about the prevention, treatment andmanagement of human problems, a number of scientists have begun to focus on thequestionofhowthisbodyofknowledgecanbetranslatedtoprovidebeneficialoutcomesforpeople.Thistaskisofhighimportancegiventhatmostscientificresearchisfundedfromthepublicpurseanditisreasonabletoexpectthattheknowledgepurchasedwilladd,insomeway,tohumanwellbeing[2,3].Recognitionoftheneedtotranslatescientificknowledge intoeffectivepolicyhas led to a general approachandbodyof knowledgethathasbeendescribedasPreventionScience.Thisfieldrepresentsarangeofmethods


271

drawnfromanumberoffieldsincludingevidence-basedmedicine,epidemiology,humandevelopmentandrelateddisciplines[4-7].

Stormont[6]summarisestheinterestsofthefieldasfollows:

Rather than simply advocating for prevention based on its own merit or intuitive appeal, prevention scientists have developed rigorous methods to help ensure that effective strategies are developed and evaluated. Among its hallmarks, prevention science emphasizes the importance of understanding the origins of emotional and behavior problems, especially modifiable risk and promotive factors; systematic intervention development targeting these factors; rigorous evaluation of intervention effects; and careful attention to implementation, dissemination, and sustainability throughout the intervention development and evaluation process. Prevention science carefully defines and embraces the value of evidence. Moreover, prevention scientists recognize the importance of ongoing local evaluation to ensure that effects observed in clinical trials persist in real world applications. (p. 1)

ThePreventionScienceapproachinvolvesasequenceofsteps;thesestepsaredesignedtoensuresystematicandrigorousdevelopment,implementationandevaluationofpolicies,programmesandinterventions.Thekeyelementsoftheapproachinvolve5basicstages:

• Stage 1 involves clear identification of the problem, disorder or issue to beaddressed.Thisstagealsorequiresthoroughreviewofwhat isknownabouttheextentoftheproblems.

• Stage2involvesreviewingtheliteratureonthefactorsandcausesassociatedwithissue(s)ofinterestusingtheavailablescientificliterature.Themostcomprehensivereviewmightincludeameta-analysisthatcombinesfindingsacrossalargenumberofstudies.

• Stage 3 involves conducting pilot studies of proposed interventions aimed ataddressingtheissuesofinterest.

• Stage4 involves large-scale, randomised controlled trials toassess theextent towhichtheproposedinterventioniseffectiveinaddressingtheissuesofinterest.

• Stage5involvestakingtheproposedinterventiontoscaleandimplementingitinthecommunityandconductingongoingevaluation.

ThesestagesareconceptualisedashavingafeedbackloopinwhichtheoutcomesfromStage 5 are used to inform further developments, thus repeating the cycle of policydevelopmentandintervention[1,8,9].

IntermsofthebasicPreventionSciencemodel,thepresentreportaddressesthefirsttwostagesby:(a)identifyinganumberofkeyissues;(b)reviewingtheevidenceonriskandprotectivefactors;and(c)identifyingpromisingevidence-basedpolicies,programmesandinterventions.

Thekeytasksthatremaintobeaddressedinvolve:

• Identifyingpolicies,programmesand interventions thatare likely tobeeffectivein aNewZealand context. Thiswill involveweighing theavailable evidence, theprogramme content and demands against a series of factors relating to theavailabilityofresourcesandfunding,issuesofculturalacceptability,thefeasibilityof implementing the programme in New Zealand contexts, and the ‘fit’ of theprogrammewithexistingpolicyframeworks.

• Developingintegratedimplementationandevaluationplansto:i)pilotanddevelopprogrammes; ii) evaluate programme efficacy using randomised controlled trials


272

orotherappropriatemethods;iii)developstrategiesfortakingtheprogrammetoscaleifitisfoundtobeeffective.

Thelaststage,takingeffectiveprogrammestoscaleisarguablythemostdifficultofthestages,especiallyiftheprogrammeistobebasedincomplexopensystemslikeschoolsorhospitals[10,11].Achievingeffectivescalerequiresovercomingatleastthreechallenges:

• Althoughprogrammeintegrityisextremelyimportant,someadaptationbythoseactuallydeliveringprogrammesisinevitable,especiallywhenprogrammesrequirejudgementsbasedondetailedknowledgeofthelocalsituationorcontext.Inordertoprotectthefidelityoftheprogrammedespitesomeadaptation,itisimportanttounderstandtheaspectsoftheprogrammethatarevitaltoitseffectiveness.

• Embedding programmes so that they are sustainable in the face of ongoingchallenges(e.g.,demographicandeconomicchanges)isalsoimportant.Embeddingaprogrammeinacontextrequiresbuildingonsitecapabilitythatwillprovidetheopportunitytomonitorandevaluatetheeffectivenessoftheprogrammeovertime.Inpartthisalsomeansbeingabletomonitorhowtheprogrammecanfitwithotheractivitiesbeingcarriedoutatthesamelocation.Inthecaseofschools,forexample,itisimportanttocutadditionalprogrammesthatdonotaddvalue.

• Stagedrolloutsothatimplementationresourcesarenotunderminedandongoingresearchanddevelopmentcanoccurintothefactorsassociatedwithsuccessinthefirsttwocomponentsisalsoimportant[1,8,9].

While the Prevention Sciencemodel provides an outline of the steps involved in thedevelopment andevaluationofpolicies, programmeand interventions therehasbeenincreasing recognition that the success of any policy, programme or intervention willdepend critically on howwell the programme is implemented as well as the intrinsicefficacy of the programmeper se. This aspect of programme implementation is oftendescribedasprogrammefidelity[4,12-14].Factorslikelytoincreaseprogrammefidelity(andthenceprogrammesuccess)include:

• Effective delivery of the programme in terms of: adherence to the programmeprinciples;participationoftheclientpopulation;responsivenessofclientstotheprogrammeandrelatedfactors.

• Organisationfactorsincluding:clearleadership;clearlinesofauthority;adequateadministrativesupportfortheprogramme;andrelatedfactors.

• Staff related factors including: staff training and competencies; adequate stafftraining;auditofstafftrainingandcompetenciesandrelatedfactors.

• Client related factors including: recognition of client needs; support to assistprogrammeparticipation;clientengagementandrelatedfactors.

• Culturalfactorsincluding:consultationwithkeygroup;determinationoftheculturalappropriatenessoftheprogramme;andrelatedfactors.

These considerations suggest that for many complex policies and programmes, theimplementationandevaluationof theprogrammewilldependcriticallyonanarrayofcomplexhumanissuesrelatingtothedeliveryoftheprogrammeandthewaysinwhichorganisational structures, staff and clients perceive the programme or policy and theculturalcontextwithinwhichtheprogrammeistobedelivered[4,12-14].Forexample,newprogrammes inschoolsneedtoconsidertherolesandresponsibilitiesofteachersand leaders within schools. In New Zealand, schools are self governing and teachersact relatively autonomously. Locally designed educational programmes such as thosereportedinChapter7haveintensivecomponentsdesignedtocapitaliseonthecapabilities


273

• prepare to test logic model or theory of change

• prepare training and implementa on procedures

1 GOOD INTENTIONS

• document interven on strategy

• prepare evalua on strategy

2 PROMISING

• test on simple evalua on with control group, using good measures and ‘reasonable’ power (not necessarily in ‘real world’)

3 EFFECTIVE

• statement of from whom, under what condi ons, and why

• prepare manual, training and coaching

• randomised controlled trial (RCT), with reasonable follow-up, good measures and sample, ideally within a system context

• arrange for replica on in another context (e.g. neighbourhood or local authority)

4 MODEL

• develop technical support to deliver interven on in ‘real life’ se�ngs• prepare a statement of evidence about the poten al causal mechanisms linking interven on to outcome• clear statement of resources necessary for interven on

�5 SYSTEM READY

• prepare procedures to monitor impact of outcomes at scale

• further RCT to include cost-benefit analysis

• encourage further independent valua on of programme expense

• prepare technical informa on for implementa on within large systems, including: − clarity of support for established prac oners such as teachers and social workers

• prepare quality assurance procedures

− clarity about likely costs, benefits and benefit realisa on

• decide whether interven on is intended to improve outcomes or has other objec ves

• agreement over target popula on

• decide realis c outcomes (using effect sizes from similar programmes)

• work out need, demand and ‘screening’

• decide proven model or innova on

• evalua on of impact of fidelity on outcomes

• agreement with experts about generalisability of interven on

• further RCT undertaken independently of programme developer

if results con nue to be promising

if results con nue to be promising

if indica ons of success

Figure 22.1: Guide to creating a system-ready evidence-based programme.ReproducedfromEarly Intervention: The Next Steps(2011),anindependentreporttoHerMajesty’sGovernment,withthekindpermissionoftheauthor,GrahamAllenMP.


274

of teachers and their leaders to take instructional decisions and be adaptive, and aregenerallylessscriptedthancomparableUSprogrammes[11].

Figure22.1providesaflowdiagramthatintegratesmuchofthematerialdiscussedaboveinto a conceptual model of the process of policy, development, implementation andevaluation using a Prevention Science approach. The diagram describes five stages ofdevelopment:

• GoodIntentions:Thefirststageoftheprocessbeginswithrecognitionoftheneedtodevelopapolicy,programme,orinterventionandaresolutionofkeyissuesrelevanttotheoutcomesoftheapplication,thetargetpopulationandrelatedmatters.

• Promising: This stage represents the translationof theGood Intentions stage todevelop the foundations of an effective policy, programme, or intervention bydeveloping an application outline, recruiting and training staff, and conductingpreliminarytestsofefficacy.

• Effective:Thisstageinvolvestestingtheefficacyoftheproposedpolicy,programmeor intervention. Ideally this should involve a well conducted and replicatedrandomisedtrial.

• Model: This stage involves formalising and further developing the proposedapplicationinreadinessforwidespreadimplementation.

• SystemReady:Thefinalstageoftheprocessinvolvesfinalisingthestepsrequiredtoimplementthepolicy,programmeorinterventiononapopulationwidebasis.

While theprevention science approachhasbeenwidely recommendedas ameansoftranslating evidence into policy, programmes and interventions, this approachhas notbeenuniformlyaccepted.Someoftheissuesthathavebeenraisedincludethecostoftheapproach,thetime-framerequiredforeffectiveevaluation,andapotentialemphasison problems rather thanon strengths [15]. In addition, theremaybe some issues forwhichtheevidencebasehasnotyetbeenestablishedandadditionalresearchandcarefulpilotingwillberequired.Inlightoftheseconsiderations,anumberofotherparadigmshavebeenproposed.AnimportantoneoftheseparadigmsintheNewZealandcontextistheTeAoMāoriperspectiveoutlinedinthenextsection.

3. A Te Ao Māori PerspectiveA feature thatpermeates the statisticsonadolescence inNewZealandare thepooreroutcomesexperiencedbyyoungMāori.Thesestatistics include:youthsuicide[16,17],mentalhealthproblems[18,19],crimeandyouthjustice[20,21]drugandalcoholmisuse[22,23],teenpregnancy[24]andotherrelatedoutcomes(seealsoChapters7and13onMāori).ReducingthedisparitiesbetweenMāoriandnonMāoriinNewZealandisthereforeamatterofthehighestsocialpriority,notonlyforaddressingadolescentproblemsbutalsoforproducingafair,equitable,andculturally-responsivesociety[25].Further,thedesireofMāoritoflourishasMāoriinAotearoa/NewZealandgivesanimperativefordevelopmentbasedonaTeAoMāoriperspectivewhichisnotcontingentonaproblemfocus.

Any discussion of these issues needs to consider the linkages between evidence andapplications that are consistentwith the Treaty ofWaitangi. The Treaty has particularrelevancetothegovernmentforthedevelopmentofsocial,healthandrelatedpoliciesandraisescomplexissuesaboutthedevelopment,designandimplementationofeffectiveprogrammesforMāori.


275

Recognition of this issue iswell represented in the recent distinctions that have beendrawn between culturally appropriate and culturally responsive programmes [26]. Ingeneralculturalappropriatenessreferstotheextenttowhichprogrammesaredevelopedand delivered in a way that is considered appropriate by Māori. On the other handculturally responsive programmes are those which more fully recognise the rights ofMāori inthedevelopment, implementationandevaluationoftheprogramme[26,27].Table22.1preparedbyMacfarlane[28]comparesandcontrastsissuesrelatingtoculturallyappropriateandculturallyresponsiveprogrammes.

AnimportantimplicationofTable22.1isthatwhileitispossibletoincorporateissuesrelatingtoculturalappropriatenessintoprogrammesdevelopedandevaluatedwithinaWesternScienceframework,thedevelopmentofculturally-responsiveprogrammesfundamentallyrequires the adoption of Māori concepts, values and world view into the process ofprogrammedevelopmentimplementation,andevaluation.Theseconsiderationshaveledto increasingadvocacybyMāori forwhathasbecometobeknownasKaupapaMāoriprogrammes[26,30-38].TheKaupapaMāoriapproachtoprogrammedevelopmenthasbeendescribedbyMacfarlaneasfollows:

The key components that define programmes as ‘kaupapa Māori’ programmes emanate from Māori worldview philosophies and perspectives, i.e.: kaupapa Māori values, beliefs, and concepts, as well as Māori-preferred processes and practices. These components serve to ‘unite’ them all as uniquely ‘Māori’, and ensure that there will be ‘cultural fit’ for those to whom they are delivered [26, 32-38]. These programmes are more likely to resonate with whānau as they draw upon the uniqueness of Māori culture, its ethos, and delivery mechanisms. The contention is that programmes must cover four fundamental areas if the service is to be sufficiently grounded so as to take on the form of kaupapa Māori.

Table 22.1. Comparison of culturally appropriate and culturally responsive policies, programmes and applications (following [28]).

Cultural appropriateness Cultural responsiveness

Referstoprogrammeselectionandcontent,i.e.:doprogrammevalues,formatandcontentalignwiththeculturalvaluesandpracticeofthetargetgroup?Itincludes:

Referstothedeliveryoftheprogrammeandtheabilitytorespondtofluid,authenticsituationsinwaysthatresonatewith(andarethereforeculturallyappropriate)andaffirmthecultureofclients.Itincludes:

Consultationwithkeygroupsinselectionprocess MāorirepresentationatagovernancelevelInspectionofprogrammecontenttodetermineaccuracy

Majorconsultationonthecontentofprogramme

Clientsatisfactionsurveys ImplementationofculturespecifictopicsStatisticalcomparisonofratesofparticipation EcologicalapproachessuchasTeWhareTapaWhā

[29]Māoriparticipationinplanningofprogrammes Afocusonwhānauora[30]Beingabletodemonstratewhānauinclusiveprinciplessuchaswhanaungatangaandmanaakitanga

IntegralMāoriprocessesandprotocolssuchaspōwhiriandwhakawhitikōrero

Aholisticapproachtotreatmentplansthataddressescultural,clinicalandwhānauneeds

Awhānauliaisonworker,advocate,therapistareintricatetotheprogramme

Anenvironmentthatcanassistinenhancingidentityandconnectionssuchasclassrooms,schoolsorgovernmentdepartments

Anenvironmentthatcanassistinenhancingidentityandconnectionssuchasmaraeortūrangawaewae,aswellasschoolsetc.

Afacilitatorwiththerightcredentials Afacilitatorwiththerightcredentials


276

Tapu. This cultural marker is concerned with the sanctity of the person; the special attributes that people are born with and that contribute to defining one’s place in time, locality and society. Often the abuse of the sanctity of the tamaiti might be caused by the erosion of Māori values, and tapu is often the corrective and coherent force that can reinstate wholeness and balance. Kaupapa Māori programmes value the sanctity of the Tamaiti.

Tikanga. This cultural marker is concerned with ‘the Māori way of doing things’. According to Mead [39] tikanga are tools of thought and understanding that are constituted to help organise behaviour and provide some predictability in how certain activities are carried out. Tikanga would include what Linda Smith identifies and explains as Māori ethics within practice [31].

Taonga tuku iho. This cultural marker is concerned with the knowledge base of mātauranga Māori – ideas, interpretations, and modifications made through generations and applicable in today’s education conundrum. Space for Māori knowledge in curricula and programmes is at the centre, not at the margins.

Tino rangatiratanga. This cultural marker is concerned with self-determination and is counter-hegemonic in the sense that curricular and programmes are expressed by Māori. Tino rangatiratanga is a dynamic construct in that it is about removing inhibitions and recognising the dignity of all who are involved in the exploration of good outcomes.

These four fundamental areas should not be considered in isolation—they coexist; they also vary together, but in patterned ways [40]. To take this perspective is to be in tune with a social constructionist approach in programme development. Such an approach will assert that these cultural markers draw from many sources and experiences that are often contrary to ‘essentialist’ formations that have been conventional traditions of thought for so long. In Māoridom, these cultural markers are not just natural or stable givens, but they have become emblematic through the ‘way of doing things’ by Māori in particular circumstances and places, over time. These fundamentals are beneficial—and therefore advantageous—for determining the distinctiveness of culturally responsive programmes.

WhiletherehasbeenincreasingadvocacyandliteratureonKaupapaMāori,theuseofthisperspectiveforthedevelopment,implementationandevaluationofprogrammesforMāorihasbeenacontestedarea.Ononehand,criticshavepointedtothepotentiallimitationsofKaupapaMāorianditspotentiallackofrigourfromthestandpointofWesternSciencealone[28].Ontheotherhand,proponentsofKaupapaMāorihavearguedfortheneedtodevelopculturallyresponsiveprogrammesandevaluationwithaTeAoMāoriframework[26,27].Further,ithasbeensuggestedthatcriticismofKaupapaMāoriandothercriticalmethodologiesfailstorecognisetheirvalueinprivilegingtheindividualandinprovidingasourceofadditionalevidenceonwhichtobasedecisions.Thesedebatesmirrorbroaderdebates about issues of rights, obligations and ownership that permeate political andsocialdebatesaboutissuesrelatingtothepositionofMāoriwithinNewZealand[41].

It is the consensus position of this report that Western Science and Kaupapa Māoriperspectives should not be seen in tension, rather an approach which encouragespartnership and cooperation between these perspectives should be taken. Variousapproachestothistaskhavebeendeveloped[42-45].KaupapaMāoriresearchprivilegestheviewsoftheparticipantswhereasscienceprivilegesthemethod.Intheend,however,bothareneeded.


277

OneapproachtoreconcilingWesternScienceandKaupapaMāoriperspectivesistheHeAwaWhiria(BraidedRiversmodel)proposedbyMacfarlane[44].ThismodelisdepictedinFigure22.2.

Thekeyfeaturesofthismodelare:

• TheWesternScienceandKaupapaMāoristreamsareacknowledgedasdistinctiveapproachestothedevelopmentandevaluationofprogrammes.

• The model permits knowledge from the Kaupapa Māori stream to inform thedevelopment of Western Science programmes and knowledge from WesternScienceprogrammestoinformthedevelopmentofKaupapaMāoriprogrammes.

• ThemodelalsopermitstheevaluationmethodologiesusedintheWesternSciencestreamtobeappliedbytheKaupapaMāoristreamandtheevaluationmethodologiesusedbyKaupapaMāoriresearchcanbeappliedtotheWesternstream.

Western scienceprogramme

Kaupapa Māoriprogramme

Western sciencegrounded evalua�on

Kaupapa Māorigrounded evalua�on

Western sciencestream

Te Ao Māoristream

Consensus on programme efficacy

Figure 22.2. Parallel streams model of western science and Kaupapa Māori programme development and evaluation.


278

• Finallythemodelassumesthattheacceptanceofprogrammesasbeingeffectivewillrelyonaconsensusbasedonknowledgefrombothstreams.

Thismodel representsapromising solution toencouraginganappropriatepartnershipbetweenWesternScienceandKaupapaMāori. Inaddition, themodelunderscores theneedtoevaluatetheimpactofprogrammesonoutcomesfordifferentparticipants.Thismeans,ontheonehand,scientificevidenceneedstobedisaggregatedsothattheeffectsofprogrammeswithdifferentgroupsindifferentcontextsareclearlyknown.Ontheotherhand,italsorequiresuseofgoodqualitativedatawhichenablesthe‘voices’ofparticipantsandtheirresponsestoprogrammestobeconsideredintheevaluation.InNewZealand,aneffectivepolicy,programme,orinterventionisonethatleadstoapositivechangeforindividualsandcollectives,whomustalsofeelthattheircultural(andother)needshavebeenvalued.

4. Encouraging systematic approaches to policy development, implementation and evaluation

Throughout theworld, changes areoccurring in thedevelopmentof social andhealthpolicies.Thesechangeshavebeenfuelledbytheincreasingbodyofresearchknowledgeabout theprogrammesandpolicies thatare likely tobeeffectiveand thosewhicharenot.Theimpetusforthisapproachhasalsobeenfuelledbyagrowingawarenessofthefactthatmanypoliciesandprogrammesandinterventionsthathavebeenrepresentedasbeingeffectivehaveprovedtobeineffectiveorevenharmful.Forexample,considertheevaluationoftheUS-based“ScaredStraight”programme[46,47].Inthisprogramme,youngmenwithahistoryofdelinquentbehavioursweretakentoprisonsandexposedtotherealitiesofprisonlifesupportedbygraphicdescriptionoftheserealitiesprovidedbyinmates.Initialqualitativeandotherevaluationssuggestedthattheprogrammewasbeneficial.Howeverwhentheapproachwassubjecttorigoroustestingusingrandomisedtrialsitwasdiscoveredthat“ScaredStraight”wasinfactharmfulwiththoseyoungmenexposedtotheprogrammebeingmorelikelytooffendthanthoseinacontrolseriesnotexposedtotheprogramme[46,47].

Findingslikethispostaclearwarningaboutthesocialandfinancialcostsofinvestinginpolicies,programmesandinterventionslackingevidenceofefficacy.Theseconsiderationsraise some important issues about the staffing and infrastructural changes that areneededtoacceleratethedevelopmentofeffectiveevidence-basedapproachestopolicydevelopment,implementationandevaluation.Theseissuesinclude:

• Theneedtobuildgreatercollaborationbetweenthescientificcommunityandpolicymakers including politicians, and keyGovernment policymakers so that the keyprinciplesofevidence-basedprogrammeselection,implementationandevaluationhavegreaterprominenceindevelopmentofkeyareasofadolescentpolicy.

• Theneedtoinvestingreatertrainingofresearchandpolicystaffintheprinciplesoftheevaluationofevidence-basedprogrammes.

• Greater training of research workers who are capable of working within bothWesternScienceandKaupapaMāoriframeworks.

• Greater recognition in public policy documents and legislation of the need toevaluatenewpolicies,programmesandinterventionsthoroughlybeforelongterminvestmentsarecontemplated.

• Thewillingnesstodiscontinueprogrammesthatarenotshowntobeeffective.


279

5. References 1. Mrazek PJ, Haggerty RJ. Reducing risks for mental disorders: frontiers for preventive

interventionresearch.1994.Washington,DC:CommitteeonPreventionofMentalDisorders,InstituteofMedicine.

2. Weber L, Bergan S, eds. The Public Responsibility for Higher Education and Research.Strasboug:CouncilofEuropePublishing;2005.

3. LernerRM,FisherCB,WeinbergRA.Towardascienceforandofthepeople:promotingcivilsocietythroughtheapplicationofdevelopmentalscience.ChildDevelopment.2000;71:11-20.

4. BotvinGJ.Advancingpreventionscienceandpractice:challenges,criticalissues,andfuturedirections.PreventionScience.2004;5:69-72.

5. KellamSG,LangevinDJ.A framework forunderstanding“evidence” inpreventionresearchandprograms.PreventionScience.2003;4:137-153.

6. StormontM, ReinkeWM, Herman KC. Introduction to the special issue: using preventionsciencetoaddressmentalhealthissuesinschools.PsychologyintheSchools.2010;47:1-4.

7. DollB,PfohlW,YoonJS,eds.HandbookofYouthPreventionScience.NewYork:Routledge;2010.

8. OldsDL,SadlerL,KitzmanH.Programsforparentsofinfantsandtoddlers:recentevidencefromrandomizedtrials.JournalofChildPsychology&Psychiatry.2007;483:4355-391.

9. MihalicS,FaganA,IrwinK,BallardD,ElliotD.Blueprintsforviolencepreventionreplications:factors for implementation success. Boulder: Institute of Behavioral Science,University ofColorado;2002.

10. CoburnC.Rethinkingscale:movingbeyondnumberstodeepandlastingchange.EducationalResearcher.2003;32:3-12.

11. CohenDK,BallDL.Educationalinnovationandtheproblemofscale.In:SchneiderB,McDonaldSK,eds.Scale-UpinEducation.Plymouth:Rowman&Littlefield;2007:19-36.

12. CenterforSubstanceAbusePrevention.Findingthebalance:programfidelityandadaptationin substance abuse prevention. Executive summary of a state-of-the-art review. 2001.Rockville:DepartmentofHealthandHumanServices,SubstanceAbuseandMentalHealthServicesAdministration.

13. Domitrovich CE, Greenberg MT. The study of implementation: current findings fromeffectiveprogramsthatpreventmentaldisordersinschool-agedchildren.JEducationalandPsychologicalConsultation.2000;11:193-221.

14. Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity ofimplementation:implicationsfordrugabusepreventioninschoolsettings.HealthEducationResearch.2003;18:237-256.

15. CatalanoRF,HawkinsJD,BerglundML,PollardJA,ArthurMW.Preventionscienceandpositiveyouthdevelopment:competitiveorcooperativeframeworks?JournalofAdolescentHealth.2002;31:230-239.

16. CoupeNM.WhakamomoriMaoriSuicidePrevention.PalmerstonNorth:MasseyUniversity;2005.

17. SkeggK,CoxB,BroughtonJ.SuicideamongNewZealandMaori:ishistoryrepeatingitself?ActaPsychiatricaScandinavica.1995;92:453-459.

18. TePuniKokiri.NgaIaoteOrangaHinengaroMaori:trendsinMaorimentalhealth:adiscussiondocument.1993.Wellington:TePuniKokiri.

19. DyallL.Maori. In:EllisPM,CollingsSCD,eds.MentalHealthinNewZealandFromaPublicHealthPerspective.Wellington:MinistryofHealth;1997:85-103.

20. DooneP.Heiwhakarurutangamateao:reportoncombatingandpreventingMaoricrime.2000.Wellington:CrimePreventionUnit,DeparatmentofthePrimeMinisterandCabinet.


280

21. MaynardK,CoeberghB,AnstissB,BakkerL,HuriwaiT.Kitearotu:towardanewassessment:the identification of cultural factors which may predispose Maori to crime. Social PolicyJournalofNewZealand.1999;13:43-54.

22. HuriwaiT.Re-enculturation:culturallycongruent interventions forMaoriwithalcohol-and-drug-use-associated problems in New Zealand. Substance Use&Misuse. 2002; 37: 1259-1268.

23. HuriwaiT,RobertsonPJ,ArmstrongD,KingiT-P,HuataP.Whanaungatanga-aprocessinthetreatmentofMaoriwithalcohol-and-drug-userelatedproblems.SubstanceUse&Misuse.2001;36:1033-1051.

24. ClarkT,RobinsonE,CrengleS,WatsonP.ContraceptiveusebyMaoriyouthinNewZealand:associatedriskandprotectivefactors.NewZealandMedicalJournal.2006;119:U1816.

25. ChappleS.Maorisocio-economicdisparity.PoliticalScience.2000;52:101-115.26. BerrymanM,WollerP,GlynnT.TheIncredibleYears:learningfromtheexperiencesofMāori

whānau andMāori staff in special education: evaluation report prepared for Ministry ofEducation,SpecialEducation.2009.Tauranga:PoutamaPounamu.

27. CherringtonL.Tehohounga:maiitetirohangaMāori:thedeliveryofconductproblemservicestoMāori.UnpublishedReport.2009.Wellington:MinistryofSocialDevelopment.

28. BlissettW,ChurchJ,FergussonD,LambieI,LangleyJ,LibertyK,PercivalT,PoultonR,StanleyP,Webster L,Werry J. Conduct problems: effective programmes for 8-12 year olds. 2010.Wellington:MinistryofSocialDevelopment.

29. DurieMH.Whaiora-MäoriHealthDevelopment.2nded.Auckland:OxfordUniversityPress;1998.

30. DurieM,CooperR,GrennellD,SnivelyS,TuaineN.WhānauOra:reportofthetaskforceonwhānau-centredinitiatives.2010.Wellington:WhānauOraTaskForce.

31. SmithLT.DecolonisingMethodologies:ResearchandIndigenousPeoples.London:ZedBooks;1999.

32. Cargo T. Māori Experiences of Delivering the Incredible Years Parenting Programme(Reflections).2008.

33. DurieMH.Whaiora:Māorihealthdevelopment.Auckland:OxfordUniversityPress;1994.34. Dionne R. Evidence-based programs in American Indian communities. 2008.Washington:

FederalInteragencyWorkGrouponChildAbuseandNeglect.35. Dionne R.Walking the good path: keeping American Indian children safe from drugs and

alcohol. http://www.unified-solutions.org/uploads/walkingthegoodpath.pdf; accessed 23February2011.

36. HerewiniT,AltenaI.IncredibleYearsMaraebasedgroup.2009.37. MacfarlaneAH.Kiahiwa rā! Listen to culture: a counternarrative to standardassessment

practicesinpsychology.TheBulletin.2008:30-36.38. Te Rau Matatini. Whiria Te Oranga: Kaumātua workforce strategy for mental health and

addictionservices.2008.PalmerstonNorth.39. MeadHM,GroveN.NgaPepehaangaTipuna:TheSayingsof theAncestors. Wellington:

VictoriaUniversityPress;2003.40. RogoffB.TheCulturalNatureofHumanDevelopment.NewYork:OxfordUniversityPress;

2003.41. MarieD,HaigBD.TheMaorirenaissanceandthepoliticizationofscienceinNewZealand.In:

OpenshawR,RataE,eds.ThePoliticsofConformityinNewZealand.Auckland:Pearson;2009:110-129.

42. DurieM.Understandinghealthand illness: researchat the interfacebetweenscienceandindigenousknowledge.InternationalJournalofEpidemiology.2004;33:1138-1145.

43. MacfarlaneS.TePikingakiRunga:raisingpossibilities.Set:ResearchInformationforTeachers.2009;2:42-50.


281

44. MacfarlaneA,BlampiedN,MacfarlaneS.Blendingtheclinicalandthecultural:aframeworkforconductingformalpsychologicalassessmentinbiculturalsettings.NewZealandJournalofPsychology.inpress.

45. Cunningham CW. A framework for addressingMäori knowledge in research, science andtechnology.PacificHealthDialog.2001;7:62-69.

46. PetrosinoA,Turpin-PetrosinoC,FinckenauerJO.Well-meaningprogramscanhaveharmfuleffects!: Lessons from experiments in Scared Straight and other like programs. Crime &Delinquency.2000;46:354-379.

47. PetrosinoA, Turpin-Petrosino C, Buehler J. ‘Scared Straight’ and other juvenile awarenessprogramsforpreventingjuveniledelinquency(UpdatedC2review).2003.Philadelphia,PA:CampbellCollaboration.

Documents

From evidence to policy, programmes and · Chapter 22: From Evidence to Policy 271. drawnom fr a number of fields including evidence-based medicine, epidemiology, human development