Lessons from reviewing and synthesising evidence on crime reduction

Lessons from reviewing and synthesising

evidence on crime reduction

What Works in Crime Reduction Conference

January 23rd 2017, British Library

Dr Lisa Tompson

l.tompson@ucl.ac.uk

UCL DEPARTMENT OF SECURITY AND CRIME SCIENCE

Lessons from reviewing and synthesising

evidence on crime reduction[ ramblings in a post-EMMIE, post-truth world ]

Overview

• The evidence landscape ground zero

• Assembling the ‘gold standard’ of the evidence base

• The ‘EMMIE-ability’ of the current crime reduction evidence

base

– At systematic review level

– At primary study level

A philosophical minefield

(So-called) evidence hierarchies

RCTs

Quasi-experimental

design

Controlled observational

studies

Observational studies (no control group)

Expert opinion

Systematic reviews

and meta-analysis

Systematic reviews

• “a review of research literature using

systematic and explicit, accountable

methods” (Gough, Oliver and Thomas, 2012:2)

• Find, sort, sift and synthesise

• Supposedly overcome the limitations of

individual studies and assess the evidence

base as a whole

Searching for SRs - inclusion criteria

• Systematic review or meta-analysis

• Measured outcome of crime prevention/

crime reduction

• Focused on a single intervention

~17,000 studies

838 studies

337

studies

82 studies

Sample (82 SRs of single interventions)

EMMIE

• Effect

– Effect direction and size

• Mechanism

– Mechanism/s activated (why it worked)

• Moderator

– Moderators/contexts that activate the mechanism/s

• Implementation

– Implementation conditions that support or obstruct delivery

• Economic

– Economic assessment of the cost effectiveness

or cost-benefit ratio of what is delivered

Effect Size (EMMIE)

Mechanisms (EMMIE)

Moderators (EMMIE)

Implementation (EMMIE)

Economics (EMMIE)

What have we learnt?

• Appraising quality is subjective automated Q-scoring

helps to standardise

• Most reviews don’t use the language of EMMIE

– Different fields have very different reporting conventions

• The evidence is generally weak on effect, and often on

other dimensions, BUT:

– Reviews can be strong on (say) moderators or implementation,

even if effect is weak

– We need to remember that reviews rely on primary study evidence

The misnomer of the ‘single intervention’

• Any intervention applied to people in the real world is

fraught with complexity and variation

After-school

programmes

to prevent

delinquency

Effect

The misnomer of the ‘single intervention’

• Any intervention applied to people in the real world is

fraught with complexity and variation

After-school

programmes

to prevent

delinquency

Effect

What’s the problem?

The misnomer of the ‘single intervention’

After-school

programmes

to prevent

delinquency

Effect

After-school

programmes

to prevent

delinquency

After-school

programmes

to prevent

delinquency

What’s in primary studies for EMMIE?

• To date we have done systematic reviews on:

Effect Size (EMMIE)

• Randomised control trials are scarce in crime prevention

– None in anti-theft tag literature

• Threats to bias are common

StudySelection

bias

Measurement

bias

Regression

to the

mean

Contamination

effects

Farrington et al. 1993 Medium Low Medium Medium

Bamfield, 1994 High Medium Unclear Medium

DiLonardo and Clarke, 1996 Medium Medium Unclear Medium

Hayes and Blackwood, 2006 Medium Low Low Medium

Beck and Palmer, 2011 Medium Medium Medium Medium

Retailer A 2015 Medium High High Unclear

Retailer B 2015 High High High Unclear

Mechanism-related information in the studies

19% Yes,

alluded to

48% Not

mentioned

• 25 studies mentioned tag-

related mechanisms

• We spoke to retailers and

loss prevention managers

to corroborate the

literature

Mechanism > Effect

Increase the

effort

Increase the risks Reduce the

rewards

Reduce

provocation

Remove excuses

1. Target harden 6. Extend

guardianship

11. Conceal

targets

16. Reduce

frustrations and

stress

21. Set rules

2. Control access

to facilities

7. Assist natural

surveillance

12. Remove

targets

17. Avoid

disputes

22. Post

instructions

3. Screen exits 8. Reduce

anonymity

13. Identify

property

18. Reduce

emotional

arousal

23. Alert

conscience

4. Deflect

offenders

9. Utilize place

managers

14. Disrupt

markets

19. Neutralise

peer pressure

24. Assist

compliance

5. Control

tools/weapons

10. Strengthen

formal surveillance

15. Deny

benefits

20. Discourage

imitation

25. Control drugs

and alcohol

Mechanism > Effect

Increase the

effort

Increase the risks Reduce the

rewards

Reduce

provocation

Remove excuses

1. Target harden 6. Extend

guardianship

11. Conceal

targets

16. Reduce

frustrations and

stress

21. Set rules

2. Control access

to facilities

7. Assist natural

surveillance

12. Remove

targets

17. Avoid

disputes

22. Post

instructions

3. Screen exits 8. Reduce

anonymity

13. Identify

property

18. Reduce

emotional

arousal

23. Alert

conscience

4. Deflect

offenders

9. Utilize place

managers

14. Disrupt

markets

19. Neutralise

peer pressure

24. Assist

compliance

5. Control

tools/weapons

10. Strengthen

formal surveillance

15. Deny

benefits

20. Discourage

imitation

25. Control drugs

and alcohol

Contextual variations that Moderate the effect

• Type of tag

– Hard vs. soft vs. ink tags

– Overt (with or without warning message) vs. discreet

• Type of shop

– Layout

– Staff availability and willingness to intervene

– Other security: signs, guards, CCTV

• Tagging strategy

– Product lines

– Dosage

– Source-tagged vs. tagged in store

Implementation > Mechanism > Effect

Increase the

effort

Increase the risks Reduce the

rewards

Reduce

provocation

Remove excuses

1. Target harden 6. Extend

guardianship

11. Conceal

targets

16. Reduce

frustrations and

stress

21. Set rules

2. Control access

to facilities

7. Assist natural

surveillance

12. Remove

targets

17. Avoid

disputes

22. Post

instructions

3. Screen exits 8. Reduce

anonymity

13. Identify

property

18. Reduce

emotional

arousal

23. Alert

conscience

4. Deflect

offenders

9. Utilize place

managers

14. Disrupt

markets

19. Neutralise

peer pressure

24. Assist

compliance

5. Control

tools/weapons

10. Strengthen

formal surveillance

15. Deny

benefits

20. Discourage

imitation

25. Control drugs

and alcohol

Conclusions

• Extracting information that speaks to EMMIE is a murky

business

• Synthesising EMMIE data requires in-depth understanding

– Of crime prevention theory

– Of evaluation theory

• Systematic review evidence depends on primary studies,

so it is crucial that we raise awareness of EMMIE

• Qualitative research is crucial for understanding the unique

qualities of interventions – we ignore their value at our peril

Thank you

Aiden Sidebottom, Amy Thornton, Lisa Tompson, Jyoti Belur,

Nick Tilley, Kate Bowers and Shane D. Johnson

University College London

l.tompson@ucl.ac.uk

UCL DEPARTMENT OF SECURITY AND CRIME SCIENCE