Sherman Evidence Based Policing

7/31/2019 Sherman Evidence Based Policing

1/16

IdeasinAmericanPolicing

ByLawrenceW.ShermanEvidence

-BasedPolicing

July1998

IdeasinAmericanPolicingpresentscommentaryandinsightfromleadingcriminologistsonissuesofinteresttoscholars,practitioners,andpolicymakers.ThepaperspublishedinthisseriesarefromthePoliceFoundationlectureseriesofthesamename.PointsofviewinthisdocumentarethoseoftheauthoranddonotnecessarilyrepresenttheofficialpositionofthePoliceFoundation.1998PoliceFoundationandLawrenceW.Sherman.Allrightsreserved.LawrenceW.ShermanisprofessorandchairoftheDepartmentofCriminologyandCriminalJusticeattheUniversityofMarylandatCollegePark.HewasthePoliceFoundationsdirectorofresearchfrom1979to1985.

POLICEFOUNDATION

Abstract

Thenewparadigmofevidence-basedmedicineholdsimportantimplicationsforpolicing.Itsuggeststhatjustdoingresearchisnotenoughandthatproactiveeffortsarerequiredtopushaccumulatedresearchevidenceintopracticethroughnationalandcommunityguidelines.Theseguidelinescanthenfocusin-houseevaluationsofwhatworksbestacrossagencies,units,victims,andofficers.Statisticaladjustmentsfortheriskfactorsshapingcrimecanprovidefaircomparisonsacrosspoliceunits,includingnationalrankingsofpoliceagenciesbytheircrimepreventioneffectiveness.Theexampleofdomesticviolence,forwhichaccumulatedNationalInstituteofJusticeresearchcouldleadtoevidence-basedguidelines,illustratesthewayinwhichagency

-basedoutcomesresearchcouldfurtherreduceviolence

againstvictims.Nationalpressuretoadoptthisparadigmcouldcomefromagency-rankingstudies,butpoliceagencycapacitytoadoptitwillrequirenewdatasystemscreatingmedicalchartsforcrimevictims,annualauditsofcrimereportingsystems,andin-houseevidencecopswhodocumenttheongoingpatternsandeffectsofpolicepracticesinlightofpublishedandin-houseresearch.TheseanalysescanthenbeintegratedintotheNYPDCompstatfeedbackmodelformanagementaccountabilityandcontinuousqualityimprovement.

Mostofushavethoughtofthestatisticiansworkasthatofmeasuringandpredicting...butfewofushavethoughtitthestatisticiansdutytotrytobringaboutchangesinthethingsthathe[orshe]measures.

W.EdwardsDeming


2/16

2

Ofalltheideasinpolicing,onestandsoutasthemostpowerfulforceforchange:policepracticesshouldbebasedon

scientificevidenceaboutwhatworksbest.Earlyinthiscentury,BerkeleyPoliceChiefAugustVollmerspartnershipwithhislocaluniversityhelpedgeneratethisidea(CarteandCarte1975),whichwasclearlyderivedfromthaterasexpansionofthescientificmethodintomedicine,management,agriculture,andmanyotherfields(Cheit1975).Whilesciencehadgreaterinitialimpactinthoseotherprofessionsduringthefirsthalfofthecentury,policinginrecentdecadeshasbeenmovingrapidlytocatchup.However,anyassessmentofthisideainmodernpolicingmustbeginwithanaccuratebenchmark:catchinguptowhat?Morecompleteevidenceonthelinkagebetweenresearchandpracticesuggestsanewparadigmforpoliceimprovementandforpublicsafetyingeneral:evidence-basedcrimeprevention.

Foryears,Sherman(1984,1992)andothershaveusedmedicineastheexemplarofaprofessionbaseduponstrongscientificevidence.Shermanhaspraisedmedicineasafieldinwhichpractitionershaveadvancedtraininginthescientificmethodandkeepup-to-datewiththemostrecentresearchevidencebyreadingmedicaljournals.Hehascitedthelargebodyofrandomizedcontrolledexperimentsinmedicinenowestimatedtonumberalmostone

millioninprint(SackettandRosenberg1995)asthehighlyrigorousscientificevidenceusedto

guidemedical

practices

.He

hassuggestedthatpolicingshouldthereforebemorelikemedicine.

Shermanwasrightabouttheneedformanymorerandomizedexperimentsinpolicing,butwrongabouthowmuchmedicinewasreally basedonscientificresearch.Newevidenceshowsthatdoctorsresistchangingpracticesbasedonnewresearchjustasmuchaspolicedo,ifnotmoreso.Closerexaminationrevealsmedicinetobeabattlegroundbetweenresearchandpractice,withusefullessonsforpolicingonnewwaystopromoteresearch.Thoselessonscomefromanewstrategycalledevidence-basedmedicine,1widelyhailedasthelong-soughtlinkbetweenresearchandpractice(Zuger1997)tosolveproblemslikethefollowing(Millenson1997,4,122,131): Anestimated85percentofmedicalpracticesremainuntestedbyresearchevidence.

Mostdoctorsrarelyreadthe2,500medicaljournalsavailable,andinsteadbasetheirpracticeonlocalcustom.

Moststudiesthatdoguidepracticeuseweak,non-randomizedresearchdesigns.

Medicine,infact,seemsjustasresistanttotheuseofevidencetoguidepracticeasarefieldswithlowereducationalre

quirements

,

suchaspolicing.TheNationalInstitutesofHealth(NIH)ConsensusGuidelinesareacaseinpoint.NIHconvenesadvisoryboardstoissuetophysiciansrecommendationsthatarebasedonintensivereviewsofresearchevidenceonspecificmedicalpractices.Theserecommendationsusuallyreceiveextensivepublicity,andarereinforcedbymailingsoftheguidelinesummariestosomeonehundredthousanddoctors.ButaccordingtoaRANDevaluation,doctorsrarelychangetheirpracticesinresponsetopublicationoftheseguidelines(Kosecoffetal.1987,ascitedinMillenson1997).Thusthreeyearsafterresearchfoundthatheartattackpatientstreatedwithcalciumantagonistsweremorelikelytodie,doctorsstillprescribedthisdangerousdrugtoone-thirdofheartattackpatients.Eightyearsafterantibioticswereshowntocureulcers,90percentofulcerpatientsremaineduntreatedbyantibiotics(Millenson1997,12325).EvidenceCops

Thestruggletochangemedicalpracticebasedonresearchevidencehasalonghistory,withvaluableimplicationsforpolicing.Inthe1840s,IgnazSemmelweissfoundevidencethatmaternaldeathinchildbirthcouldbereducedifdoctors

1 Thetermevidenceinthismono-graphreferstoscientific,notcriminal,evidence.


3/16

3

washedtheirhandsbeforedeliveringbabies.HethentriedtoapplythisresearchtomedicalpracticeinVienna,whichledtohisbeingdrivenoutoftownbyhisboss,thechiefobstetrician.Hundredsofthousandsofwomendiedbecausetheprofessionrefusedtocomplywithhisevidence-basedguidelinesforsomefortyyears.Thestoryshowstheimportantdistinctionbetweenmerelydoingresearchandattemptingtoapplyresearchtoredirectprofessionalpractices.Onewaytodescribepeople

whotrytoapplyresearchistheroleofevidencecop.MorelikeatrafficcopthanVictorHugosdetectiveJavert,theevidencecopsjobistoredirectpracticethroughcomplianceratherthanpunishment.Whilethisjobmaybeaschallengingasherdingcats,itstillconsistsofpointingprofessionalstopracticethisway,notthatway.Asinallpolicing,thesuccessrateforthisjobvarieswidely.Fortunately,theinitialfailuresofpeoplelikeSemmelweisspavedthewayforgreatersuccessinthe1990s.

ConsiderScottWein-garten,M.D.,ofCedars-Sinai

HospitalinLosAngeles.AsdirectorofthehospitalsCenterforAppliedHealthServicesResearch,Weingartenisanevidence-cop-in-residence.Hisjobistomonitorwhatthe2,250doctorsaredoingtopatientsatthehospitalandtodetectpracticesthatruncountertorecommendationsbasedonresearchevidence.Hedoesthisthroughproddingratherthanpunishment,conveninggroupsofdoctorswhotreatspecificmaladiestodiscusstheresearchevidence.Thesegroupsthenproducetheirownconsensusguidelinesforpracticesthatbecomehospitalpolicy.Thirty-fivesuchsetsofguidelineswereproducedinWeingartensfirstfouryearsonthejob(Millenson1997,120).WhatNIH,Weingarten,and

the1995foundersofthenewjournalcalledEvidence-BasedMedicinearealltryingtodoistopushresearchintopractice.Justaspolicinghasbecomemoreproactiveatdealingwithcrime,researchersarebecomingmoreproactiveaboutdealingwithpractice.Thistrendhasdevelopedinmanyfields,notjustmedicine.

Increasedpressureforreinventinggovernmenttofocusonmeasurableresultsisreflectedinthe1994U

.S.

GovernmentPerformanceResultsAct(GPRA),whichrequiresallfederalagenciestofileannualreportsonquantitativeindicatorsoftheirachievements.Educationisundergrowingpressuretoraisetestscoresasproofthatchildrenarelearning,whichhasledtoincreaseddiscussionofresearchevidenceonwhatworksineducation(Raspberry1998).AndtheU.S.Congresshasrequiredthattheeffectivenessoffederallyfundedcrimepreventionprogramsbeevaluatedusingrigorousandscientificallyrecognizedstandardsandmethodologies(House1995,sec.116).Allthissetsthestageforanewparadigmformakingresearchmoreusefultopolicingthanithaseverbeenbefore.

KeyQuestionsInsuggestinganewparadigm

calledevidence-basedpolicing,therearefourkeyquestionstoanswer:Whatisit?Whatisnewaboutit?Howdoesitapplytoaspecificexampleofpolicepractice?Howcanitbeinstitutionalized?

Whatisit?Evidence-basedpolicingisthe

useofthebestavailableresearchontheoutcomesofpoliceworktoimplementguidelinesandevaluateagencies,units,andofficers.Putmoresimply,

Onewaytodescribepeoplewhotrytoapplyresearchistheroleofevidencecop.


4/16

4

evidence-basedpolicingusesresearchtoguidepracticeandevaluatepractitioners.Itusesthebestevidencetosha

pethebest

practice.Itisasystematicefforttoparseoutandcodifyunsystematicexperienceasthebasisforpolicework,refiningitbyongoingsystematictestingofhypotheses.

Evaluationofongoingoperationshasbeenthecrucialmissinglinkinmanyrecentattemptstoimprovepolicing.Ifitistruethatmostpoliceworkhasyettogobeyond911(Sparrow,Moore,andKennedy1990),theunderlyingreasonmaybealackofevaluationsystemsthatclearlylinkresearch-basedguidelinestooutcomes.Itisonlywiththatadditionthatpolicingcanbecomeareflexiveorsmartinstitution,continuouslyimprovingwithongoingfeedback.

Thebasicpremiseofevidence-basedpracticeisthatweareallentitledtoourownopinions,butnottoourownfacts.Yetleftalonetopracticeindividually,practitionersdocomeupwiththeirownfacts,whichoftenturnouttobewrong.Arecentsurveyof82WashingtonStatedoctorsfound137differentstrategiesfortreatingurinarytractinfections(Berg1991).Nodoubtthesameresultcouldbefoundforhandlingdomesticdisturbances.Astudyevaluatingtheaccuracyofstrepthroatdiagnosesbasedonunstructuredexaminationbyexperiencedpediatriciansfoundit

farinferiortoasystematic,evidence-basedchecklistusedbynurses.Themythicpowerofsub

jectiveandunstructured

wisdomholdsbackeveryfieldandkeepsitfromsystematicallydiscoveringandimplementingwhatworksbestinrepeatedtasks.

Aprimeexampleofthepowerofsystematic,ongoingevaluationscomesagainfrommedicine.In1990,theNewYorkStateHealthDepartmentbegantopublishdeathratesforcoronarybypasssurgerygroupedbyhospitalandindividualsurgeon.Thisactionwaspromptedbyresearchshowingthatwhilethestatewideaveragedeathratewas3.7percent,somedoctorsranashighas82percent.Moreover,afteradjustingfortheriskofdeathbythepre-operationconditionofthepatientcaseload,patientswere4.4timesmorelikelytodieinsurgeryattheleastsuccessfulhospitalsthanatthebesthospitals.Despiteenormousoppositionfromhospitalsandsurgeons,thesedataweremade

public,revealingastrongpracticeeffect:themoreoperationsdoctorsandhospitalsdideachyear

,thelowertherisk-ad

justed

deathrate.Usingthisclearcorrelationtopushlow-frequencysurgeonsandhospitalsoutofthisbusinessaltogether,hospitalswereabletolowerthedeathrateintheseoperationsby40percentinjustthreeyears(Millenson1997,195).

Evidence-basedpolicingisabouttwoverydifferentkindsofresearch:basicresearchonwhatworksbestwhenimplementedproperlyundercontrolledconditions,andongoingoutcomesresearchabouttheresultseachunitisactuallyachievingbyapplying(orignoring)basicresearchinpractice.Thiscombinationcreatesafeedbackloop(fig.1)thatbeginswitheitherpublishedorin-housestudiessuggestinghowpolicingmightobtainthebesteffects.Thereviewofthisevidencecanleadtoguidelinestakinglaw,ethics,andcommunity

Figure1.Evidence-BasedPolicing.Literature

Best

Evidence

In-House

Guidelines

Outputs

Outcomes


5/16

5

cultureintoaccount.Theseguidelineswouldspecifymeasurableoutputs,orpracticesthat

policeareaskedto

follow.Theirvaryingdegreesofsuccessatdeliveringthoseoutputscanthenbeassessedbytrackingrisk-adjustedoutcomes,orresultsoverareasonablylongfollow-upperiod.Theseoutcomesmaybedefinedinseveraldifferentways:offensesper1,000residents,repeatvictimizationsper100victims,repeatoffendingper100offenders,andsoon.Theobservationthatsomeunitsaregettingbetterresultsthanotherscanbeusedtofurtheridentifyfactorsassociatedwithsuccess,whichcanthenbefedbackasnewin-houseresearchtorefinetheguidelinesandraisetheoverallsuccessleveloftheagency.Suchresearchcouldalsobepublishedinnationaljournalsoratleastkeptinanagencydatabaseasinstitutionalmemoryaboutsuccessandfailureratesfordifferentmethods.Whatisnewaboutit?

Skepticsmaysaythatthereisnothingnewinevidence-basedpolicing,andthatotherparadigmsalreadyembracetheseprinciples.Oncloserexamination,however,wewillseethatnootherparadigmcontainstheprinciplesforitsownimplementation.Nootherparadigmcontainsaprincipleforbothchangingpracticesandmeasuringthesuccessofthosechangeswithrisk-adjusted

outcomesresearch(likebypasssurgerydeathrates).NootherparadigmnotevenNYPDsComputerizedCrimeComparisonStatistics(Compstat)strategy(BrattonwithKnobler1998)usesscientificevidencetoholdprofessionalsaccountableforresultsinpeer-reviewedandevenpublicdiscussionsofoutcomesevidence.

Evidence-basedpolicingis

clearlydifferentfrom,butveryhelpfulto,allthreepresentparadigmsofpolicing.Incident-specificpolicing,or911responses,currentlylackanyoutcomesmeasureexcepttimeoutofservice.Policeofficerswhotaketoomuchtimetohandleacallaresometimesaccusedofshirkingandareurgedbysupervisorstoworkfaster.2Butnoonetrackstherateofrepeatcallsbyofficerorunittoseehoweffectivethefirstresponsewasinpreventingfutureproblems.Evidence-basedpolicingcould

usesuchoutcomestojustifylongertimespentoneachcallonthebasisofanofficersaverageresults,ratherthanissuingacrudedemandthatheorshestaywithinanaveragetimelimit.Itcouldalsoplacemuchmoreemphasisonlearninghowtodealwitheachcallmosteffectivelyandpreventively,aquestionthatcurrentlygetslittleattention.

Communitypolicing,howeverdefined,isnotclearlylinkedtoevidenceabouteffectivenessinpreventingcrime.Itismuchmoreabouthowtodopoliceworkasetofoutputsthanitisaboutdesiredresults,oroutcomes.Workingwiththecommunityandlisteningtoandrespectingcommunitymembersareallimportantelementsoftheparadigm.Butthatparadigmalonehasbeeneasyformanyofficerstoignore.Addingtheaccountabilitysystemsfromtheparadigmofevidence-basedpolicingcouldactuallymakepolicefarmoreactiveinworkingwiththecommunity.

Problem-orientedpolicingisclearlythemajorsourcefor

2 Thissoundsoddlylikethepressurefordrive-in,drive-outchildbirthhealthinsurancenowbarredbyfederallaw.

Evidence-basedpolicingisclearlydifferentfrom,butveryhelpfulto,allthreepresentparadigmsofpolicing.


6/16

6

evidence-basedpolicing.HermanGoldsteinswritings(1979,1990),aswellasJohnEckandWilliamS

pelmansSARAmodel

(1987),clearlyemphasizeassessmentofproblem-solvingresponsesasakeypartoftheprocess.Yetthereisnoclearstatementabouttheuseofscientificevidenceeitherinselectingstrategiesforrespondingtoproblemsorinmonitoringtheimplementationandresultsofthosestrategies(Sherman1991).Reportsonproblem-orientedpolicinghavesofarproducedlittleevidenceeitherfromcontrolledtestsoroutcomesresearch.Becausetheparadigmstressestheuniquecharacteristicsofeachcrimepattern,problem-orientedpolicinghasnotbeenusedtorespondtohighlyrepetitivesituationslikedomesticassaultsordisputes.Fewcomparisonsofdifferentmethodsforattackingthesameproblemhavebeendeveloped.Fewofficersareevenheldaccountablefornotimplementingaproblem-solvingplantheyhaveagreedtoundertake.Problem-orientedpolicinghasclearlyrevolutionizedthewaymanypolicethinkabouttheirobjectives,movingthemawayfromanarrowfocusoneachincidenttoabroaderfocusonpatternsandsystems.Butintheabsenceofpressurefromanevidence-basedapproachtoevaluatingsuccessandmanagementaccountability,problem-orientedpolicinghasbeenkeptatthemarginsofpolicework.

NYPDsCompstatstrategy(BrattonwithKnobler1998)haspushedtheresultsaccountabilityprinci

plefartherthaneverbefore

,

butithasnotusedthescientificmethodtoassesscauseandeffect.Successfulmanagersarerewarded,butsuccessfulmethodsarenotpinpointedandcodified.Whatevidence-basedpolicing

addstotheseparadigmsisanewprinciplefordecisionmaking:scientificevidence.Mostpolicepractice,likemedicalpractice,isstillshapedbylocalcustom,opinions,theories,andsubjectiveimpressions.Evidence-basedpolicingchallengesthoseprinciplesofdecisionmakingandcreatessystematicfeedbacktoprovidecontinuousqualityimprovementintheachievementofpoliceobjectives(seeHoover1996).Hencetheinspirationforthisparadigmisnotonlymedicineanditsrandomizedtrials,butalsotheprinciplesofqualitycontrolinmanufacturingdevelopedbyWalterShewhart(1939)andW.EdwardsDeming(1986).TheseprincipleswereinitiallyrejectedbyU.S.businessleaders,butwerefinallyembracedinthe1980safterJapaneseindustriesusedthemtofarsurpassU.S.manufacturersinthequalityoftheirproducts.Whatmakesbothpolicing

andmedicinedifferentfrommanufacturing,ofcourse,isthefargreatervariabilityintherawmaterialtobeprocessedhumanbeings.Thatiswhatgivesthegoldstandardofevaluationresearch,therandomized

controlledtrial,bothitsstrengthanditslimitations.Thestrengthoftheresearchdesign,pioneeredin

policin

gb

ythePolice

Foundation,isitsabilitytoreduceuncertaintyabouttheaverageeffectsofapolicyonvastnumbersofpeople.Thelimitationoftheresearchdesignisthatitcannotescapevariabilityintreatments,responses,andimplementation.

Thevariabilityoftreatmentsinpolicingismuchlikethatinsurgery,whichstandsinsharpcontrasttopharmaceuticals.Whilethechemicalcontentofmedicaldrugsisalmostalwaysidentical,theproceduralcontentofsurgeryvarieswidely.Similarly,thestyleandtoneeachofficerbringstoacitizenencountervariesenormouslyandcanmakeabigdifferenceintheoutcomeofaspecificcase.Dosage,timing,andfollow-upofbothdrugsandpoliceworkcanvarywidelyinpractice.

Evenholdingtreatmentconstant,thereisevidencethatbothpatientsandoffendersrespondtotreatmentswithwidevariations.Someoftheseresponses,allergicreactions,cankillsomepeoplewithtreatmentsthatcuremostothers.Offendersareknowntovaryintheirresponsestopoliceactionsbyindividual,neighborhood,andcity.Andimplementationofnewpracticesbasedoncontrolledexperimentsinbothmedicineandpolicingvariesaccordingtohowwellresearchiscommunicated,howmuchinformationiscreated


7/16

7

aboutwhetherpracticesactuallychange,andhowmuchreinforcementthereisforthechan

ge,both

positiveand

negative.

Evidence-basedpolicingassumesthatexperimentsalonearenotenough.Puttingresearchintopracticerequiresjustasmuchattentiontoimplementationasitdoestocontrolledevaluations.Ongoingsystemsforresearchingimplementationcanclosethefeedbacklooptocreatetheprincipleofindustrialqualityimprovement.

Howdoesitapplytoaspecificexampleofpolicepractice?

Thepolicingofdomesticviolenceoffersaclearillustrationofwhatisnewabouttheevidence-basedparadigm.Domesticviolencehasbeenthesubjectofmorepolicepracticesresearchthananyothercrimeproblem.Theresearchhasarguablyhadlittleeffectonpolicepractice,atleastbythenewstandardsofevidence-basedmedicine.Yettheavailableevidenceoffersafairandscientificallyvalidapproachforholdingpoliceagencies,units,andofficersaccountablefortheresultsofpolicework,asmeasuredbyrepeateddomesticviolenceagainstthesamevictims.

TheNationalInstituteofJustice(NIJ)andthePoliceFoundationhaveprovidedpolicingwithextensiveinformationonwhatworkstopreventrepeatedviolence.The

researchhasalsoshownthat,likesurgery,policepracticesvarygreatlyintheirimplementation.Thesevariationsin

practicecause

varyingresultsforrepeatoffendingagainstvictims.Evenholdingpracticeconstant,responsestoarrestvarybyoffender,neighborhood,andcity.Finally,researchshowsverypoorcompliancewithmandatoryarrestguidelinesaftertheyareadopted(Ferraro1989).

Therearemanyvarietiesofarrestformisdemeanordomesticviolence.Theoffendermayormaynotbehandcuffed,arrestedinfrontoffamilyandneighbors,givenachancetoexplainhisversionofeventstothepolice,or

treatedwithcourtesyandpoliteness.Dothesevariationsonthethemeofarrestmakeadifference?The

yshould

,

accordingtothedefiancetheoryofcriminalsanctioneffects(Sherman1993).AndtheydidinMilwaukee,accordingtoRaymondPaternosterandhiscolleagues(1997).TheMilwaukeeevidencerevealsthatcontrollingforotherriskfactorsamongsome800arrestedoffenders,thosewhofelttheywerenottreatedinaprocedurallyfairandpolitemannerwere60 percentmorelikelytocommitareportedactofdomesticviolenceinthefuture(fig.2).Thisfindingsuggeststhreeways

0%

10%

20%

30%

40%

40%

50%

25%

Fair Unfair

Figure2.RepeatDomesticViolenceandPoliceFairness.

Source:Paternoster,etal.


8/16

8

topushresearchintopractice:1) changetheguidelinesformakingdomesticviolencearreststoincludethoseelementsthatwouldenableoffenderstoperceivemoreproceduraljustice;2)holdpoliceaccountableforusingtheseguidelinesbycomparingratesofrepeatvictimizationassociatedwithdifferentpoliceunits;and3) computetheseratesusingstatisticaladjustmentsforthepre-existinglevelofrecidivismrisks.

TheNIJresearchprovidesotherevidenceforwaysthatpolicecanreducerepeatoffendinginmisdemeanordomesticviolence.Ratherthanaone-size-fits-allpolicy,theevidencesuggestsspecificguide-linestobeusedunderdifferentconditions.Offenderswhoareabsentwhenpolicearriveastheyareinsome40 percentofcasesrespondmoreeffectivelytoarrestwarrantsthanoffenderswhoarearrestedonthescene(Dunford1990).Offenderswhoareemployedaredeterredbyarrest,whileoffenderswhoareunemployedgenerallyincreasetheiroffendingmoreiftheyarearrestedthaniftheyarehandledinsomeotherfashion(PateandHamilton1992;Berketal.1992;ShermanandSmith1992).Offenderswholiveinurbanareasofconcentratedpovertycommitmorerepeatoffensesiftheyarearrestedthanifnot,whileoffenderswholiveinmoreaffluentareascommitfewerrepeatoffensesiftheyarearrested(Marciniak1994).Allofthese

findingscouldbechangedbyfurtherresearch,butforthemomenttheyarethebestevidenceavailable

.

Thisresearchevidencecouldsupportguidelinesforpolicingdomesticviolencethatdifferedbyneighborhoodandabsenceorpresenceoftheoffender.Itcouldalsosupportguidelinesaboutlisteningtosuspectssideofthestorybeforemakingarrestdecisionsandgenerallytreatingsuspectswithcourtesy.Otherevidence,suchastheextremelyhigh-riskperiodforrepeatvictimizationinthefirstdaysandweeksafterthelastpoliceencounter(StrangandSherman

1996),couldbeusedtofashionnewproblem-orientedstrategies.Mostimportant,theexistingresearchcanbeusedtocreateafairsystemforevaluatingpoliceperformanceonthebasisofrisk-adjustedoutcomes.Thatevidence(fig.3)showsthatthelikelihoodofarepeatoffenseisstronglylinkedtothenumberofpreviousoffenseseachoffenderhas.Oncetheriskofrepeat

offendingcanbepredictedwithreasonableaccuracy,itbecomespossibletousethosepredictionsasabenchmarkforpoliceperformance.JustasinthebypasssurgerydeathratesinNewYork,theoutcomesofpolicingcanbe

Figure3.RiskofRepeatDomesticAssaultbyPriors.

MilwaukeeDomesticViolenceExperiment

0

20

40

60

80

0 1 2 3 7

PercentRepeats

42%

48%

75%

60%


9/16

9

controlledfortherisklevelinherentinthecaseloadtheyface.Usingacitywidedatabaseofalldomesticassaults,nowrunningovertenthousandcasesperyearincitieslikeMilwaukee,amodelcanbeconstructedtoassesstheriskofrepeatoffendingineachcase.Theoverallmixofcasesineachpoliceprecinctorforeachofficercangenerateanaveragerisklevelforthatcaseload.Eachpolicepatroldistrictcanthenbeevaluatedaccordingtotheactualversuspredictedrateofrepeatoffendingeachyear(fig.4).Allpatroldistrictsinthecitycanthenbecomparedonthebasisoftheirrelativepercentagedifferencebetweenexpectedandactualratesofrepeatdomesticassault(fig.5).

Byconstructinginformationsystemsforthiskindofoutcomeresearch,policedepartmentscanfocusonanob

jectivethathas

onlypreviouslybeenmeasuredinmajorexperiments.Makingthegoalofpolicingeachdomesticassaulttheoutcomeofareducedrepeatoffendingrateratherthanthe outputofwhetheranarrestismadewouldhaveseveraleffects.Oneisthatcrimepreventionwouldgetgreaterattentionthanretributionforitsownsake.Whilenoteveryonewouldwelcomethat,itisconsistentwithatleastsomepoliceleadersviewofthepurposeofthepoliceasacrimepreventionagency(BrattonwithKnobler1998).Anothereffectwouldbetoseekoutand

Figure4. Observedvs.ExpectedRiskofRepeatDomesticViolence.

0

10

20

30

40

50

60

Observed Expected

PercentRepeat

25%

50%

-100

-50

0

50

100

150

200

PCT1 PCT2 PCT3 PCT4 PCT5PercentRepeat

Figure5.Observedvs.ExpectedRankingbyPrecinct.

50%

25%

50%

150%


10/16

10

eveninitiatemoreresearchonwhatworksbesttopreventdomesticviolence.Intheworldaswenowknowit,nooneinpolicingfromthepolicechieftotherookieofficerhasanydirectincentivetoreducerepeatoffendingagainstknownvictims.Nooneinpolicingisheldaccountableforaccomplishing,orevenmeasuring,thatobjective.Asaresult,nooneknowswhetherrepeatvictimizationratesgetbetterorworsefromyeartoyear.Usingoutcomesevidencetoevaluateperformancewouldmakepolicepracticesfarmorevictim-centered,thetopprioritybeingthatofpreventinganyfurtherassaults.

Howcanitbeinstitutionalized?

Thestrongestclaimaboutevidence-basedpolicingisthatitcontainstheprinciplesofitsownimplementation.Theprinciplesofusingevidencebothtochangeandevaluatepracticecanbeappliedtoabroadinstitutionalanalysisofimplementation.Thuswhilethechangesdescribed

abovewouldhavetooccuronepoliceagencyatatime,therearecertainnationalforcesthatcanhelpstarttheballrolling.Thiscanbeseen,forexample,innationalrankingsofbig-citypoliceagencies,aswellasnationalmandatesforimprovingpolicedatasystemstoprovidebetterevidence.Yetevensuchexternalpressureswillnotsucceedwithoutinternalevidencecopstoimport,apply,andcreateresearchevidence.

Noinstitutionislikelytoincreasevoluntarilyitsaccountabilityexceptunderstrongexternalpressure.Itisunlikelythatevidence-basedpolicingcouldbeadoptedbyapoliceexecutivesimplybecauseitappearstobeagoodidea.Thehistoryofevidence-basedmedicineandeducationstronglysuggeststhatprofessionalswillonlymakesuchchangesunderexternalcoercion.Nothingseemstofostersuchpressureasmuchasperformancerankingsacrossagencies(Millenson1997;Steinberg1998).Justasvariouspublicperformancemeasures

allowstockbrokerstorankpublicly-heldcorporationsandprovidethosecompanieswithstron

gincentivesforbetter

results,publicinformationaboutpoliceperformancewouldcreatethestrongestpressureforimprovement.3

Oneexampleofhowthemajorcitypolicedepartmentscouldberankedonperformancecanbefoundintheirhomiciderates,whichalreadyreceiveextensivepublicity.Whatthesestatisticslack,however,isanyscientificanalysisofexpectedrisk.Policeperformancehasnothingtodo,atleastintheshortrun,withthesocial,economic,demographic,anddrugmarketforcesthathelpshapeacityshomiciderate.Whilepoliceperformancemayalsoaffectthosehomiciderates,theotherfactorsmustbetakenintoaccount.Usingrisk-adjustedhomicideratesprovidesoneindicationofhowwellapolicedepartmentmaybedoingthingslikeconfiscatingillegalweapons,patrollinghotspots,regulatingviolenttavernsanddrugmarkets,andmonitoringyouthgangs.Whilethebasicresearchliteraturewouldincreasinglyprovideasourceofguidancefortakinginitiativesagainsthomicide,a

3 The1919resultsofthefirstnationalrankingsofhospitalsweredeemedsothreateningthattheAmericanCollegeofSurgeonsdecidedtoburnthereportimmediatelyinthefurnaceofNewYorksWaldorf-AstoriaHotel(Millenson1997,146).

Thestrongestclaimaboutevidence

-basedpolicingisthat

itcontainstheprinciplesofitsownimplementation.


11/16

11

risk-adjustedoutcomesanalysis(fig.6)wouldindicatehowwellthatresearchhadbeenputintopractice

.

4

Ifacrediblenationalresearchorganizationwouldproducesuchleaguerankingsamongbig-citypolicedepartmentseachyear(likethe U.S.News&WorldReportrankingsofcollegesanduniversities),thepredictableresultintheshorttermwouldbeattacksonthemethodologyused.Thatis,infact,whatcontinuestogooninNewYorkwiththedeathratesinsurgery.ButtheNewYorkrankingshavespreadtootherstates,andconsumershavefoundthemquitevaluable.Doctorsandpolicemayalsofindrankingsveryvaluableinthelongrun.Bothprofessionsshouldenjoygreaterpublicrespectastheygetbetteratproducingtheresultstheirconsumerswant.

Themoreseriouslyperformanceindicatorsinfluencethefateoforganizations,themorelikelytheyaretobesubverted.RecentexamplesincludetheU.S.PostalServiceinWestVirginia,whereanelaborateschemetodefeattheon-timemaildeliveryauditwasrecently

alleged(McAllister1998).Otherexamplesincludeteachershelpingstudentstocheatontheiranswerstonationalachievementtestsand

,

ofcourse,policedepartmentsunder-reportingcrime.TheNewYorkCitypolicehaveremovedthreecommandersinthepastfiveyearsforimproperlycountingcrimetomaketheirperformancelookbetter(Kocieniewski1998),andseveralchiefsofpoliceelsewherehavebeenconvictedoncriminalchargesforsimilarconduct.

Quiteapartfrompressurestocorruptdata,criminologistshavelongknownthatpolicecrimereportingisnotreliable,withthe

possibleexceptionofhomicide.Notwoagenciesclassifycrimethesameway.Thesameeventma

ybecalledana

ggravated

assaultinoneagencyandamiscellaneousincidentinanother.TherecentFBIdecisiontodropPhiladelphiafromthenationalcrimereportingprogramwasnotanisolatedaction.In1988,theFBIquietlydroppedtheentirestatesofFloridaandKentucky.SincetheFBIlacksresourcestodoon-siteauditsineachpoliceagencyeveryyear,theseexamplesarejustthetipofaverybigiceberg.Therearealreadyrisingsuspicionsofpolicemanipulationofcrimedataas

4WhilemanyofthebasicriskfactorswouldbecomputedfromCensusdatathatcouldbeoutofdatebythemiddleofeachdecade,otherriskdatacanbederivedfromannuallyupdatedsources,suchastheNIJADAMdataondrugabuseamongarrestees.Unemployment,schooldropout,teenchildbirth,andinfantmortalitydataarealsoavailableannuallyforeachcityandcouldhelppredicttheexpectedrateofhomicide.

HypotheticalData

Figure6.HomicidebyCity,Actualvs.Predicted.

-60

-40

-20

0

20

40

60

80

NYC Balt. Chi LA Dallas

PercentDifference

50%

25%

25%

60%

NYC Baltimore Chicago LA Dallas


12/16

12

crimeratesfallinmanycities.Moreseriouspressurefromnationalrankingswouldthreatendatainte

grit

yevenmore

.

OneviablesolutiontothisproblemisafederalrequirementforpolicedepartmentstoretainCPAfirmstoproduceannualauditsoftheirreportedcrimedata.Thisrequirementcouldbeimposedasaconditionforreceivingfederalfunds,justasmanyotherfederalmandateshavealreadydone.Anticipatingcourtchallengesaboutunfundedmandates(suchastheBradyBill),Congresscouldalsoprovidefundstopayfortheaudits.CrimecountingstandardscouldbesetnationallybytheaccountingprofessionincollaborationwiththeFBI.Alternatively,eachstatelegislaturecouldrequire(orevenfund)theseauditsasameansofassuringfairnessinperformancerankingsofpolicedepartmentswithinthestate.Stateagenciessuchasthecriminaljusticestatisticalcenterscouldalsoproducesuchrankingsasaservicetotaxpayers.Statesalreadyhavetheoptionofspendingfederalfundsonsuchapurposeunderthebroadcategoryofevaluationfunds.

Intheprocessofrevitalizingcrimedataintegrity,therewouldbegreatvalueinreorganizingpolicedatasystems.Mostimportantwouldbethecreationofamedicalchartforeachcrimevictim.Likecomputerizedpatientrecords,thischartwouldshowthediagnosis(offense

description)foreachincidentavictimpresentstoapoliceagency,perhapsanywhereinthestate.Thechartwouldalsoshowwhatpolicedidinresponse,everythingfromtakinganoffensereporttoarrestinganoffenderwhosereleasedatefromprisonisalsokept,updated,inthecomputerizedvictimchart.Thisinformationtoolcouldhelpdevelopmanyproactivepolicemethodsforpreventingrepeatvictimization.Allowingofficerstousethesedatatokeeptheirownprivatebattingaveragesforrepeatvictimization(evenwithoutadjustingforrisk)mayencouragethemtobecomeinvolvedandcommittedtodoingabetterjobatpreventingcrime.

Betterrecordsarealsoneededaboutwhatpolicedoaboutcrimeaccordingtocertainpatternsofoffenses.Medicalchartsforviolenttaverns,frequentlyrobbedconveniencestores,andotherhotspotswheremostcrimeoccurswouldbeveryusefulforongoingproblem

-

orientedpolicing

attemptstoreducerepeatoffendingatthoseplaces.Similarrecordscouldbekeptaboutapatternofcrimesspreadoutacrossawiderarea,suchasautomatictellermachinerobberies.Ifofficerteamsorunitsidentifytheseplacesorpatternsascrimetargetsanddesignateacontrolgroup,thesemedicalchartscanbecomethebasisforestimatinghowmuchcrimeeachpoliceunithasprevented.

Computerscanalsohelppoliceofficerstoimplementpracticeguidelines.Medicalcomputersystemsnowofferrecommendedpracticeguidelinesinresponsetoachecklistofdata,aswellaswarningwhendrugprescriptionsfalloutsideprogrammedparametersofdiseasetypeanddosage.Theuseofhand-heldcomputerstoadviseofficersinthefieldandtoprovideinstantqualitycontrolchecksmaynothappensoon,butthegrowthofpoliceresearchmaymakeitinevitableinthelongrun.Doctorsarenotexpectedtokeep

Intheprocessofrevitalizingcrimedataintegrity,therewouldbegreatvalueinreorganizingpolicedatasystems.


13/16

13

largeamountsofresearchdataintheirheads,norevenmedicalguidelinesforeachdiagnosis.Com

puterswillnotre

place

good

judgment,buttheycanclearlyenhanceit.

Federalrulescouldalsorequirepolicedepartmentstoappointacertifiedpolicecriminologist(eitherinternallyorinpartnershipwithauniversityorresearchorganization),whowouldbecometheagencysevidencecop.LikeScottWeingartenofCedars-Sinai,thedepartmentalcriminologistwouldberesponsibleforputtingresearchintopractice,thenevaluatingtheresults.Whetherthecriminologistisactuallyanemployeeorauniversityprofessorworkinginpartnershipwiththepolicemaynotmatterasmuchastheroleitself.Thecriminologistcouldhelpdevelopmoreeffectiveguidelinesforpreventingrepeatoffending,andcoulddevelopexpectedversusactualrepeatoffendingdatabyoffensetypeforeachpolicedistrictordetectiveunit.AcriminologistcouldaddthescientificmethodtotheNYPDCompstatprocess(BrattonwithKnobler1998),providingstatisticsateachmeetingoneachpatroldistrictscrimetrendsandpatterns(orevenitscomplaintsagainstpoliceofficers)inrelationtothedistrictsrisklevel.Buildingthecapacitytoimport,apply,andcreateevidencewithineachpoliceagencymaybeanessentialingredientinthesuccessofthisparadigm.

Wemayalsofindthatthetraditionaldistancebetweenresearchersandpoliceofficialsshrinkswhenresearchers

provide

moreimmediatemanagerialinformation.Criminologistshavelongrefusedtoprovidepolicemanagerswithdataonparticularofficers,deemingitcontrarytotheethicsofbasicresearch(Hartnett1998).Byfinallyprovidingthedatainascientificallyreasonableformat,criminologistsmaybecomefarmoreeffectiveatpushingresearchintopractice.

Criminologistscanalsoactonthefindingthatdoctorstendtochangepracticesbasedonpersonalinteractionandrepeatedcomputerizedfeedback,andnotfromconferences,classes,orwrittenresearchreports(Millenson1997,12730).Similarfindingshavebeenpublishedabouttheeffectivenessofagriculturalextensionservices,

inwhichuniversityscientistsvisitfarmsandshowfarmersnewtechniquesforimprovingtheircro

py

ields.The

yechoaChinese

proverb:TellmeandIwillforget;showmeandIwillremember;involvemeandIwillunderstand.

TheonetestofthisprincipleinpolicingtodateisAlexWeisss(1997)researchonhowpolicedepartmentsadoptinnovations.Basedonanationalsurveyofpolicechiefsandtheirtopaides,Weissdiscoveredthattelephonecallsfromagencytoagencyplayedavitalroleinspreadingnewideas.Whilewrittenreportsmayhavesupplementedthephonecalls,word-of-mouthseemstobethemajorwayinwhichpoliceinnovationsarecommunicatedandadopted.Weisssstudysuggeststhe

greatimportanceofgatheringmoreevidenceonevidence.Theempiricalquestionforresearchis,whatpracticesworkbesttochangepractices?Thisinherentlyreflexiveposturemayleadustoempiricalcomparisonsoftheeffectivenessof,forexample,NIJconferences,massmailingsofresearch-in-briefreports,ornewone

-

on-

oneapproaches.OneexampleofthelatterwouldbeproactivetelephonecallstopoliceagenciesaroundtheU.S.madebypresentorformerpoliceofficers;callerscouldbetrainedbyresearchorganizationstodescribenewresearchfindings.Ifnationalconsensusguidelinesforpracticeweredevelopedbypanelsofpoliceexecutivesand

Theempiricalquestionforresearchis,whatpracticesworkbesttochange

practices?


14/16

14

researchers,thecallerscouldcommunicatethoseaswell.Otherapproachesworthtestingmightincludefielddemonstrationsinpolicetechnique.Thistrainingwouldnotbebasedonexperience,asisthecurrentFieldTrainingOfficersystem,butratheritwouldbebasedonevidencethatthemethodbeingdemonstratedhasbeenproveneffectiveinreducingrepeatoffending.

ConclusionThetestofthisparadigms

resultsisnotwhetheritisadoptedthisyearorintwentyyears.AsLordKeyneshassuggested,theinfluenceofideasmaybefarmoreglacialthanvolcanic.Thepressureforbettermeasuresofresultsisinthespiritoftheage,andpolicecannotlongescapeit.Allthispaperdoesisaddoneinchtotheglacier,sothatwecansayofpolicingwhatDr.WilliamMayooftheMayoClinicsaidofhisprofessionalmostacenturyago:Thegloryofmedicineisthatitisconstantlymovingforward,thatthereisalwayssomethingmoretolearn.

References

Berg,A.O.1991.Variationsamongfamilyphysiciansmanagementstrategiesforlowerurinarytractinfectionsinwomen:AreportfromtheWashingtonPhysiciansCollaborativeResearchNetwork. Journalofthe

AmericanBoardofFamilyPractice(SeptemberOctober):32730.

Berk,RichardA

.;AlecCam

pbell

;

RuthKlap;andBruceWestern.1992.Thedeterrenteffectofarrestinincidentsofdomesticviolence:ABayesiananalysisoffourfieldexperiments. American

SociologicalReview57:698708.

Bratton,William,withPeterKnobler.1998.Turnaround:HowAmericastopcopreversedthecrimeepidemic.NewYork:RandomHouse.

Carte,Gene,andElaineCarte.1975. PolicereformintheUnitedStates:TheeraofAugustVollmer.Berkeley:UniversityofCaliforniaPress.

Cheit,Earl.1975.Theusefulartsandtheliberaltradition.NewYork:McGraw-Hill.

Deming,W.Edwards.1986.Outofthecrisis.Cambridge:MassachusettsInstituteofTechnology,CenterforAdvancedEngineeringStudy.

Dunford,Franklyn.1990.System-initiatedwarrantsforsuspectsofmisdemeanordomesticassault:Apilotstudy.JusticeQuarterly7:63153.

Eck,John,andWilliamSpelman.1987. Problem-solving:

Problem-orientedpolicinginNewportNews.Washington,D.C.:PoliceExecutiveResearchForum.

Ferraro,KathleenJ.1989.Policingwomanbattering.SocialProblems36:6174.

Goldstein,Herman.1979.Improvingpolicing:Aproblem-orientedapproach.CrimeandDelin

quenc

y25

:

23658.

.1990.Problem-orientedpolicing.NewYork:McGraw-Hill.

Hartnett,Susan.1998.AddresstotheThirdNationalInstituteofJusticeConferenceonPolice-ResearchPartnerships,February.

Hodge,MelvilleH.1990.DirectusebyphysiciansoftheTDSMedicalInformationSystem.In AHistoryofMedicalInformatics,editedbyBruce I.BlumandKarenDuncan.NewYork:ACM.

Kosecoff,Jacqueline,etal.1987.EffectoftheNationalInstitutesofHealthConsensusDevelopmentProgramonphysicianpractice. JournaloftheAmericanMedicalAssociation258(November20):270813.

McAllister,Bill.1998.AspecialdeliveryinWestVirginia:Postalemployeescheattobeatratingsystem.WashingtonPost,10January,A1.

Marciniak,Elizabeth.1994.Communitypolicingofdomesticviolence:Neighborhooddifferencesintheeffectofarrest.Ph.D.diss.,UniversityofMaryland.

Millenson,MichaelL.1997.Demandingmedicalexcellence:Doctorsandaccountabilityin


15/16

15

theinformationage.Chicago:UniversityofChicagoPress.

OfficeofTechnologyAssessmentoftheCon

gressoftheUnited

States.1983.Theimpactofrandomizedclinicaltrialsonhealthpolicyandmedicalpractice.BackgroundpaperOTA-BP-H-22.Washington,D.C.:GovernmentPrintingOffice.

Pate,AntonyM.,andEdwinE.Hamilton.1992.Formalandinformaldeterrentstodomesticviolence:TheDadeCountySpouseAssaultExperiment. American

SociologicalReview57:69198.

Paternoster,Ray;BobbyBrame;RonetBachman;andLawrenceW.Sherman.1997.Dofairproceduresmatter?ProceduraljusticeintheMilwaukeeDomesticViolenceExperiment. LawandSocietyReview.

Raspberry,William.1998.Tried,trueandignored.WashingtonPost,2February,A19.

Sackett,DavidL.,andWilliamM.C.Rosenberg.1995.On

theneedforevidence-basedmedicine. HealthEconomics4:24954.

Sherman,LawrenceW

.1984

.

Experimentsinpolicediscretion:Scientificboonordangerousknowledge?LawandContemporaryProblems47,no.4:6181.

.1992.Policingdomesticviolence:Experimentsanddilemmas.NewYork:FreePress.

.1993.Defiance,deterrenceandirrelevance:Atheoryofthecriminalsanction. JournalofResearchinCrimeandDelinquency30:44573.

andDouglasA.Smith.1992.Crime,punishmentandstakeinconformity:Legalandinformalcontrolofdomesticviolence.AmericanSociologicalReview57.

Shewhart,WalterA.1939.Statisticalmethodsfromtheviewpointofqualitycontrol.EditedbyW.E.Deming.Lancaster,Pennsylvania:GraduateSchooloftheU.S.DepartmentofAgriculture.

Sparrow,Malcolm;MarkMoore;andDavidKennedy.1990.Beyond911:Aneweraforp

olicing

.NewYork:Basic

Books.

Steinberg,Jacques.1998.Publicshaming:Ratingsystemforschools:Somestatesarefindingthathumiliationleadstoimprovement.NewYorkTimes,7January,A19.

Strang,Heather,andLawrenceW.Sherman.1996.Predictingdomestichomicide.PaperpresentedtotheAmericanAssociationfortheAdvancementofScience,January,inBaltimore,Maryland.

U.S.Congress.House.104thCongress,1stsess.,H.Rept.104-387,sec.116.

Weiss,Alexander.1992.Diffusionofinnovationsinpolicedepartments.Ph.D.diss.,NorthwesternUniversity.

Zuger,Abigail.1997.Newwayofdoctoring:Bythebook.NewYorkTimes,16December,C1.


16/16

16

1201ConnecticutAvenue,NW,Washington,DC20036(202)833-1460Fax:(202)659-9149e-mail:[email protected]

ABOUTTHEPOLICEFOUNDATION

ThePoliceFoundationisa

private,in

de

pendent,not-for-

profitor

ganization

de

dicate

dto

supportinginnovationandimprovementinpolicingthroughitsresearch,technicalassistance,andcommunicationsprograms.Establishedin1970,thefoundationhasconductedseminalresearchinpolicebehavior,policy,andprocedure,andworkstotransfertolocalagenciesthebestnewinformationaboutpracticesfordealingeffectivelywitharangeofimportantpoliceoperationalandadministrativeconcerns.Motivatingallofthefoundationseffortsisthegoalofefficient,humanepolicingthatoperateswithintheframeworkofdemocraticprinciplesandthehighestidealsofthenation.

BOARDOFDIRECTORSChairman

WilliamG.MillikenPresident

HubertWilliamsFredaAdler,PhDLeeP.Brown,PhDWilliamH.HudnutIIIW.WalterMenninger,MD

VictorH.PalmieriHenryRuth

StanleyK.SheinbaumAlfredA.Slocum

SallySuchilKathrynJ.Whitmire

POLICEFOUNDATION

OFFICEOFRESEARCHDavidWeisburd,PhD

SeniorResearchScientistRosannGreenspan,PhD

ResearchDirectorPatrickR.Gartin,PhD

SeniorResearchAssociateDavidG.Olson,PhD

SeniorResearchAssociateEdwinE.Hamilton,MASeniorResearchAnalystMichaelClifton,MAResearchAssociate

JenniferC.Nickisch,MAResearchAssociateJustinReady,MAResearchAssociateAnnetteC.Miller,MAResearchAssistantRachelDadusc,BA

AdministrativeAssistant

Documents

Sherman Evidence Based Policing