* Karen Bullock, Department of Sociology, University of Surrey, Guildford, Surrey, UK. E-mail:
** NickTilley,UCLJill Dando Institute ofCrime Science,UniversityCollege London, London,UK.
E-mail:[email protected]
1
Evidence-Based Policing and Crime Reduction
Karen Bullock∗ and Nick Tilley∗∗
Abstract There have been calls for research evidence to be drawn into police practice. We
examine evidence-based practice in the policing and crime reduction agenda, drawing
on the experience of implementing problem-oriented policing in the UK and beyond.
We suggest that that the development of such an agenda has been hampered by certain
factors. Evidence is not routinely used by police officers (or partnerships) developing
strategies to deal with crime problems who prefer to deliver traditional (law
enforcement) responses. There is a limited knowledge base on which practitioners can
draw in developing responses to crime problems, and the nature of evidence about
what is effective is contested amongst academics. Whilst welcoming the moves to
incorporate evidence in policing, we caution against excessive optimism about what
can be achieved and make some recommendations for those engaged in developing
evidence-based practice.
Introduction
Alongside other areas of public policy and practice, there have been calls for research
‘evidence’ to be drawn in policing (ACPO/Centrex, 2006; Flanagan’s Review of
Policing, Policing Green Paper, 2008). The critical collection, analysis and
presentation of evidence are, of course, part of the stock-in-trade of policing.
Evidence is used, for example, to determine if an offence has been committed, to
suggest who might have committed it and to try to prove a case beyond reasonable
doubt in a court of law. But this is not what is being referred to in ‘evidence based
policing’. Rather, ‘evidence-based policing’ is analogous to evidence-based medicine
(see Nutley et al., 2002; Davies et al., 2000). As with its medical counterpart,
evidence-based policing refers to the application of measures on the basis of robust
evidence of their effectiveness in dealing with real (rather than supposed) problems.
As Sherman says (1998: 3–4), ‘evidence-based policing uses research to guide
practice and evaluate practitioners. It uses the best evidence to shape the best
practice.’
Evidence-based policing assumes that we should adopt a sceptical attitude
towards traditional ways of working for which there is no systematic evidence of
effectiveness. The evidence-based movement in medicine has cast doubt on
conventional wisdom; for example, making babies lie on the sides or stomachs, by
looking for hard evidence about its actual effects rather than its assumed effects. It
turns out that far fewer cot deaths occur where babies are put on their backs and this is
now how new parents are advised to place their children in cribs and cots. In just the
same way, evidence-based policing begins by calling for empirical tests of both
established and innovative policies and practices.
Various developments in policing have emerged as efforts to deliver evidence-
based policing, amongst which the National Intelligence Model (NIM) is one clear
example. This paper focuses, however, on problem-oriented policing (POP). It does
so in part because POP has a longer history than the NIM, in part because there has
been more research on the implementation of POP than of the NIM, in part because
POP has enjoyed a greater presence than the NIM in countries other than the UK
(although the NIM may soon overtake it), and in part because POP is more
wholeheartedly committed to participating in the evidence-based public service
reform agenda than the NIM. Indeed, Sherman (1998) argues that problem-oriented
policing is the major source of evidence-based policing emphasizing assessment of
problem-solving responses as a key to the process.
In the UK, POP has over the past five years metamorphosed from problem-
oriented policing to problem-oriented partnership, and hence its fate as a vehicle for
developing and delivering evidence based responses to crime problems has a
significance that includes but extends beyond the work of statutory police services.
‘Problem-oriented partnerships’ describe larger or smaller groups (including statutory
police services, but extending beyond them) that aim to reduce, ameliorate or remove
significant community crime-related crime and disorder issues that it is the
responsibility and/or interest of members to address them. The distinctive
methodology of problem-oriented policing and partnership ostensibly involves the
exploration, application and creation of evidence directed at the delivery of effective
responses to real problems of community concern. The experience of efforts to adopt
problem-oriented policing and partnership over the past three decades is used here as
a basis for exploring the prospects and possibilities for making use of and benefitting
from an evidence-based approach to policing more generally.
The theory of evidence use in POP
By far, the best-known model for problem-oriented policing and partnership is SARA:
Scanning, Analysis, Response and Assessment. SARA has been criticized for
oversimplifying quite complex and difficult processes that rarely follow a simple
linear path from one stage to the next. Yet SARA captures well the underlying logic
and highlights the central role evidence use and evidence generation are supposed to
play in POP. Scanning involves assembling evidence on the basis of which sustained
specific problems of community concern are identified. Analysis involves efforts to
identify necessary conditions for the problem to persist with a view to focusing
responses on those conditions most open to practical, cost-effective long-term change.
It also involves taking a critical look at past responses to the problem to try to work
out what has not worked and why---remembering that the problems focused on are
those that have been found to remain in spite of previous practices to deal with them.
Response describes well-focused efforts to target interventions that will reduce the
specific problem. It involves casting around for past well-researched effective
responses to similar problems, and drawing on that evidence. And assessment refers
to the cool-headed, objective evaluation of the implementation and outcome
effectiveness of the strategies put in place in order to enable appropriate lessons to be
learned and passed on to others. Assessments thereby most clearly contribute to the
evidence base, although scanning and analysis may also do so as problems are more
clearly defined and their sources understood. Scanning, analysis and assessment all
involve the collation and systematic analysis of data. They entail a form of action
research, not unlike that used, at its best, in evidence-based health care.
Although not explicitly couched in problem oriented terms, a variety of
initiatives relating to repeat victimization illustrate the potential of POP. Scanning in
the Kirkholt estate revealed that repeat incidents accounted for a high proportion of
the total number of burglaries (see Forrester et al., 1988, 1990) (and later work
building on Kirkholt has shown that the patterns found there are found in pretty much
all places and for pretty much all offences). Analysis revealed that the offences tended
to occur quickly, that properties targeted tended to be insecure and that prepayment
meters were especially attractive targets (and later work has again built on this to
1 A US and UK award scheme, respectively, which recognizes excellence in problem-oriented policing.
show that repeat incidents tend to be undertaken by the same offenders or associates
who know what they can expect if they commit a further offence). Responses have
been tailored to the analyses, including in Kirkholt, implementing rapid security
upgrades, removal of prepayment meters and the formation of mini-neighbourhood
watches (‘cocoon watch’) around burgled properties. And assessment has focused on
the precise expected changes in light of the specific problem and the measures
adopted to deal with it, which in Kirkholt involved looking, for example, at the
overall change in the burglary rate as well, specifically, as the contribution of repeat
incidents to the figures.
The appeal of POP is rather obvious. In addition to some evidence of its
outcome effectiveness both as an overall approach (Weisburd et al., 2008) and in
individual projects (Goldstein and Tilley award entries),1 it chimes well with the
broader evidence based agenda and the adoption of ‘scientific’ methods in addressing
social issues more generally.
Evidence use in POP in practice
POP’s brainchild was Herman Goldstein, who first proposed it in 1979 and went on to
a book length manifesto in 1990. In 2003, he delivered a devastating verdict on
policing as he saw it, where he stated that
Th(e) bare-bones premise - that operations should be informed by knowledge -
is so elementary and so self-evident, that one feels apologetic in drawing
attention to it. It is not an especially controversial proposition. It is certainly
not radical. And it is not novel . . . (A)part from what is reflected in the case
studies in problem-oriented policing and the substantial rhetoric associated
with the concept, there is no discernable, sustained and consistent effort within
policing to make the basic premise that ‘knowledge informs practice’ a routine
part of policing. (Goldstein, 2003, pp. 19–20)
Goldstein refers here to ‘knowledge’ rather than ‘evidence’. The term
‘knowledge-based policing’ has become widely used (see Williamson, 2008) and
refers similarly to the use of research, intelligence and technology to inform policing.
It suffices to note however that Goldstein’s remarks cast doubt on police progress in
making a reality of his vision, and the same would go for partnerships more widely.
Indeed, Goldstein’s conclusions reflect those reached by many who have
looked systematically at efforts to implement POP, especially in the UK (Leigh et al.,
1998; Read and Tilley, 2000; Townsley et al., 2003; Bullock et al., 2006) and the
USA (Scott, 2000). There is little use of evidence in deciding what to do about
problems. Instead, there is a tendency to fall back on conventional law enforcement
responses. For example, Leigh et al. (1998) found that direct police-led actions
against alleged offenders were most favoured by officers:40%of all the actions
recorded by Cleveland and Leicestershire police services were traditional police
responses of patrol, observations, surveillance, warrants, arrests and warnings.
Matassa and Newburn (2003), in a study of the implementation of problem-oriented
hate crime projects in the Metropolitan Police Service, note that police officers
perceived deficiencies in the ability of partner agencies to implement interventions
and so they tended to resort to traditional policing methods of improved high-
visibility patrol and surveillance of offenders. Read and Tilley (2000), in an
examination of 266 problem-oriented initiatives returned as part of an HMIC
inspection, found that the most common source of advice in developing new
2 Randomized control trials are experimental methods aimed at identifying cause and effect. A feature
is random allocation to intervention groups. Measurements are made before and after treatment and any
differences that are found in the before and after measurements between the groups are attributable to
the treatment. They are often referred to as the ‘gold standard’ of the evaluation design but practical
and ethical problems abound. 3 Systematic reviews synthesize the findings of a number of studies examining the effectiveness of a
particular intervention. They set out the criteria for including or excluding studies, the search strategy,
how the studies were analysed, and should it be possible, they include meta-analyses of the findings.
responses to problems was colleagues in the same force (42%) or a different force
(43%). Townsley et al. (2003) conducted a small-scale survey of officers and found
that 58% (n = not stated) had not read a Home Office research report in the six
months prior to the survey. Bullock et al. (2006), in a review of 150 entries to the UK
Tilley Award, similarly found that projects rarely indicated that they had made use of
the existing literature about how to address problems when developing their projects.
In total, 41 of the 150 projects analysed referred to the research or literature that
provided additional information about responses with a small number (n = 8) referring
to multiple sources.
Explanations for the neglect of evidence
The good sense of making use of ‘knowledge’ or ‘evidence’ of effectiveness by
designing strategies to deal with knotty problems appears to be pretty obvious to all.
Why, then, has it not happened? There is not necessarily a single cause. Indeed there
are several possibilities that are not mutually exclusive, as we shall see. Possible
explanations fall into four main types: those relating to the nature of evidence; those
relating to the availability of evidence; those relating to the constraints on
organizations and those relating to the nature of the issues to be addressed. Before
coming to these, it is important to point out that the difficulties that have been found
in delivering on the evidence-based policy and practice agenda are not confined to
policing and crime prevention (Davies and Nutley, 2002; Moseley and Tierney, 2005).
The nature of evidence
What counts as ‘evidence’ of effective practice is a tricky issue and in practice has
been considered to be very broad (Solesbury, 2001), from small-scale qualitative
studies to large-scale social experiments. In health care, a hierarchy of evidence for
assessing evidence about effectiveness has been developed with randomized control
trials (RCTs)2 or better yet systematic reviews
3 of these at the pinnacle (Nutley et al.,
2002).Whilst there may be general acceptance of experimental methods in
determining what is effective, what counts as evidence of success is hotly debated in
the criminal justice field (Davies et al., 2000). Reflecting those used in health care,
hierarchies of evidence have been developed (see Sherman, 1998; Sherman et al.,
2002; Harper and Chitty, 2005) but the utility of these have been questioned (see for
example Hollin 2008 and Tilley, 2009). Of particular significance, the case for RCTs
has not been accepted in the criminal justice arena in a straightforward manner
(Nutley and Davis, 2000) by either practitioners or researchers. Nutley and Davis
(2000) point to concerns raised by practitioners (a) that if an intervention is perceived
to be effective, it may be unethical to withhold it and (b) that if the impact is
perceived to be doubtful, there may be public protection issues. Arguments about
whether these objections are justified notwithstanding, RCTs are rare in the policing
and crime reduction field. Farrington and Welsh (2005) describe 12 policing RCTs,
none of which were conducted in the UK, and most of which examined arrest
strategies for domestic violence perpetrators and targeting crime hotspots.
The availability of evidence
4 www.crimereduction.homeoffice.gov.uk/toolkits/.
5 www.popcenter.org.
6 www.campbellcollaboration.org.
Concerns have been expressed regarding the honest and timely publication of
government-funded research in the UK. Hope (2004), in commenting on the
presentation of the findings of a government sponsored burglary reduction initiative,
pointed to ‘denial, pretence and the selective presentation and use of evidence in
support of political expediency’ (Hope, 2004). (See also Raynor (2008).) Sherman
(2009) refers to the 2008 Ministry of Justice publication of the findings of a £5
million RCT examining outcomes of restorative justice programmes in England and
Wales. Sherman (2009) suggests that despite containing useful lessons for both policy
and practice, the results of this research have not been well publicized because they
do not fit into the prevailing dominating political ideology.
For some time, authors have pointed to the importance of a collective body of
knowledge on how to tackle crime problems as a means of facilitating the delivery of
problem-oriented responses (Hoare et al., 1984; Scott, 2000; Irving and Dixon, 2002;
Eck, 2003; Goldstein, 2003; Townsley et al., 2003). Goldstein (1990) (like others)
argued that police officers do not have access to a body of evidence about the
behaviours and problems that they are expected to deal with and that in these
circumstances it is hardly surprising that the police often do not know what to do
when dealing with problems.
Attempts have been made to systematically disseminate academic evidence about
what is effective in tackling crime problems in order that practitioners can draw on
what is known to be effective. An early effort was conducted by Sherman et al.
(1997); their influential paper ‘What Works, What Doesn’t and What’s Promising’
was commissioned by the US congress. In the UK, a review of existing evidence
regarding what is known to be effective in reducing reoffending was conducted as
part of the 1998 Comprehensive Spending Review (Goldblatt and Lewis, 1998)
(updated in part by Harper and Chitty in 2005).
The Home Office has developed ‘tool kits’4 that are a web-based resource
containing advice about how to tackle a range of crime problems based on problem-
solving methodology. However, these Home Office websites do not appear to be
regularly updated and the extent to which they are used is unclear (Townsley et al.,
2003). The ‘POP centre’5 is a US web-based resource containing large amounts of
material regarding what techniques are effective in reducing crime and in what
circumstances, along with materials regarding the implementation of POP.
The Campbell Collaboration6 publishes systematic reviews of research
evidence in the field of (amongst others) criminal justice. At the time of writing, the
Campbell library contained 14 systematic reviews of ‘crime’ interventions, though it
is likely to grow. However, systematic reviews have to draw on good quality research.
It has already been argued that the evidence base about what is effective about
tackling crime problems is limited. Indeed, in their systematic review of the impact of
POP, the authors noted surprise about the small numbers of studies that met their
inclusion criteria (10 out of about 5,500 articles and reports) (Weisburd et al., 2008).
This calls into question the nature of the conclusions they can make from the evidence:
‘our results suggest problem-oriented policing has a modest impact on reducing crime
and disorder, but we urge caution in interpreting these findings, because of the small
number of eligible studies we located and the diverse group of problems and response
these studies included’ (Weisburd et al., 2008). A policy maker or practitioner
looking at this review for help in decision making may be left feeling none the wiser.
Organizational constraints
Moseley and Tierney (2005) identify certain obstacles to the translation of evidence
into practice. Organizations typically do not give practitioners time to search for and
read the research literature that, if it takes place at all, may be confined to the
weekend or the evening. Reading new research on effective practice may not be
perceived as ‘real work’ and doing so may be seen as a ‘threat’ to professional
judgement and expertise. Additionally, rewards - in terms of career progression - are
generally not based on the development of specialism and expertise in a chosen field
and may be a disincentive to incorporate research evidence into practice. All this
would equally apply to policing.
Conclusion
The call for research evidence to be routinely drawn into practice is widely endorsed
and its appeal seems obvious. There is a broad agenda for evidence-based practice in
public policy, but there are widespread obstacles to its realization. Many of these
obstacles apply to any evidence-based agenda but some are more specific to policing.
First, there is limited evidence about the effectiveness of interventions aimed at
tackling the kinds of problems that the police are commonly asked to address. Second,
the evidence that is available may not be presented in a manner that is readily
accessible to police officers and their partners. Officers may face additional
difficulties in sifting ‘robust’ from weak or otherwise misleading evidence. Third,
complicating this there are, in any case, competing notions of what ‘doing’ evidence
based work should look like. Lastly, there are difficulties in changing the traditional
practice habits of police officers and other professionals working in the field.
There remains then a need for the police service (and the research community)
to pay attention to factors that may facilitate the development of evidence-based
practice in this area. These include
1. a commitment to the generation of evidence about the impact of interventions
in the policing and crime reduction field. There is also a need to facilitate the
production of wider knowledge, for example, the production of knowledge
about the process and practice of doing crime prevention work;
2. a need to inculcate the habits of evidence-based practice more routinely into
police practice; for example, the habits of engaging with the research literature,
thinking about current policing practice in light of that research and keeping
an open mind about practice and processes;
3. a commitment to recognizing and supporting progress and good practice in
this area and;
4. a need to manage expectations about what can be achieved and how quickly it
can be achieved, as some disappointment is inevitable.
References
ACPO/Centrex. (2006). Practice Advice on Professionalizing the Business of
Neighbourhood Policing. Centrex: Wyboston.
Bullock, K., Erol, R., and Tilley, N. (2006). Problem-Oriented Policing and
Partnership. Cullompton: Willan.
Clarke, R. (1998). “Defining Police Strategies.” In Shelley, T. and Grant, A. (eds),
Problem Oriented Policing: Crime-Specific Problems, Critical Issues and Making
POP Work. Washington, DC: Police Executive Research Forum.
Davies, H. and Nutley, S. (2002). “Evidence Based Policy and Practice - Moving
from Rhetoric to Reality.” Research Unit for Research Utilisation Discussion Paper.
Davies,H., Nutley,S., and Smith, C. (eds) (2000) What Works? Evidence-Based Policy
and Practice in Public Services. Bristol: Policy Press.
Eck, J. (2003). “Police Problems: The Complexity of Problem Theory, Research and
Evaluation.” In Knutsson, J. (ed.), Problem-Oriented Policing: From Innovation to
Mainstream.Crime Prevention Studies, vol. 15. Cullompton: Willan.
Farrington, D. and Welsh, B. (2005). “Randomized Experiments in Criminology:
What Have We Learned in the Last Two Decades?” Journal of Experimental
Criminology 1:1–29.
Flanagan, R. (2008). An Independent Review of Policing. London: Home Office.
Forrester, D., Chatterton, M., and Pease, K. (1988). The Kirkholt Burglary Prevention
Project, Rochdale. Crime Prevention Paper 13. London: Home Office.
Forrester, D., Frenz, S., and O’Connell, M. (1990). The Kirkholt Burglary Prevention
Project, Phase 2. Crime Prevention Unit Paper 23. London: Home Office.
Goldblatt, P. and Lewis, C. (eds) (1998). Reducing Offending: An Assessment of
Research Evidence on Ways of Dealing with Offending Behaviour. Home Office
Research Study 187. London: Home Office.
Goldstein, H. (1979). “Improving Policing: A Problem-Oriented Approach.” Crime
and Delinquency 25(2): 236–258.
Goldstein, H. (1990). Problem-Oriented Policing. New York: McGraw Hill.
Goldstein, H. (2003). “On Further Developing Problem-Oriented Policing.” In
Knutsson, J. (ed.), Problem-Oriented policing: From Innovation to Mainstream.
Crime Prevention Studies, vol. 15. Cullompton: Willan .
Harper, G. and Chitty, G. (2005). The Impact of Corrections on Re-offending. Home
Office Research Study 291. London: Home Office.
Hoare, M., Stewart, G., and Purcell, C. (1984). The Problem-Oriented Approach:
Four Pilot Studies. Metropolitan Police Management Services Department Report
Number 30/84. London: Metropolitan Police Service.
Hollin, C. (2008). “Evaluating Offender Behaviour Programmes: Does Only
Randomisation Glitter?” Criminology and Criminal Justice 8(1): 89–106.
Hope, T. (2004). “Pretend it Works: Evidence and Governance in the Evaluation of
the Reducing Burglary Initiative.” Criminal Justice 4(3): 287–308.
Irving, B. and Dixon, B. (2002). Hotspotting. London: Police Foundation.
Leigh, A., Read, T., and Tilley, N. (1998). Brit Pop II: Problem-Oriented Policing in
Practice. Police Research Series Paper 93. London: Home Office.
Matassa, M. and Newburn, T. (2003). “Problem-Oriented Evaluation? Evaluating
Problem-Oriented Policing Initiatives.” In Bullock, K. and Tilley, N. (eds), Crime
Reduction and Problem-Oriented Policing. Cullompton: Willan.
Moseley, A. and Tierney, S. (2005). “Evidence Based Practice in the Real World.”
Evidence and Policy 1(1): 113–119.
Nutley, S. and Davis, H. (2000). In Davies, H., Nutley, S., and Smith, C. (eds), What
Works? Evidence-Based Policy and Practice in Public Services. Bristol: Policy Press.
Nutley, S., Walter, I., and Davies, H. (2002). From Knowing to Doing: A Framework
for Understanding the Evidence into-Practice Agenda. Research Unit for Research
Utilisation Discussion Paper 1. St Andrews: University of St Andrews.
Raynor, P. (2008). “Community Penalties and Home Office Research: On the Way
Back to ‘Nothing Works’?” Criminology and Criminal Justice 8(1): 73–87.
Read, T. and Tilley, N. (2000). Not Rocket Science? Problem Solving and Crime
Reduction. Crime Reduction Research Series Paper 6. London: Home Office.
Scott, M. (2000). Problem-oriented Policing: Reflections on the First 20 Years.
Washington, DC: Office of Community Oriented Policing Services, US Department
of Justice.
Sherman, L. (1998). Evidence Based Policing. Washington, DC: Police Foundation.
Available at www.policefoundation.org/pdf/Sherman.pdf.
Sherman, L. (2009). “Evidence and Liberty: The Promise of Experimental
Criminology.” Criminology and Criminal Justice 9(1): 2–28.
Sherman, L. W., Farrington, D. P, Welsh, B. C., and MacKenzie, D. L. (eds). (2002).
Evidence-Based Crime Prevention. London: Routledge.
Sherman, L. W., Gottfredson, D., MacKenzie, D. L., Eck, J., Reuter, P., and Bushway,
S. (eds) (1997). Preventing Crime: What Works, What Doesn’t, What’s Promising.
Report to the US Congress. Washington, DC: National Institute of Justice, Office of
Justice Programs, US Department of Justice.
Solesbury, W. (2001) Evidence Based Policy: Whence it Came and Where It’s Going.
ESRC UK Centre for Evidence Based Policy and Practice Working Paper 1. London:
ESRC UK Centre for Evidence Based Policy and Practice.
Tilley, N. and Laycock, G. (2002). Working Out What to Do: Evidence-Based Crime
Reduction.Crime Reduction Research Series Paper 11. London: Home Office.
Tilley, N. (2009). Sherman vs. Sherman: Realism vs. Rhetoric. Criminology and
Criminal Justice 9(2): 135–144.
Townsley, M., Johnson, S., and Pease, K. (2003). “Problem orientation, Problem
Solving and Organisational Change.” InKnuttson, J. (ed.),Crime Prevention Studies,
vol. 15.New York: Criminal Justice Press.
Weisburd, D., Telep, C., Hinkle, J., and Eck, J. (2008). “The Effects of Problem-
Oriented Policing on Crime and Disorder.” Campbell Systematic Reviews 2008: 14.
Williamson, T. (ed.) (2008). The Handbook of Knowledge-Based Policing. Chichester:
Wiley.
Recommended
