i

Domestic Violence Perpetrator

Programme: Social Return on Investment

Analysis

Steven Ariss, Parveen Ali, Hazel Squires and Elizabeth Goyder

Economic Evaluation

ii

DECLARATION OF INTERESTS This analysis was conducted by a research team at the University of Sheffield. The report is

primarily for commissioners and delivering organisations of domestic violence and abuse

(DVA) perpetrator programmes with an aim to share good practice, measure outcomes

effectively and demonstrate Social Return on Investment (SROI). The report is available to key

partners such as the local authority, public health commissioners, perpetrator programme

delivering organisations and academics.

The research was funded by the NIHR School for Public Health Research (SPHR) Public Health

Practice Evaluation Scheme (PHPES)1. The views expressed are those of the author(s) and not

necessarily those of the NHS, the NIHR or the Department of Health.

Suggested Citation:

Ariss, S., Ali, P.A. Squires, H. (2017). Domestic Violence Perpetrator Programme: Social Return

on Investment Analysis. University of Sheffield

An electronic copy of this report and other Project publications are available at:

https://www.sheffield.ac.uk/snm/research/doncaster

For any information or questions, please contact:

Dr. Steve Ariss (s.ariss@sheffield.ac.uk)

Dr. Parveen Ali (parveen.ali@sheffield.ac.uk)

1 The NIHR School for Public Health Research is a partnership between the Universities of Sheffield, Bristol, Cambridge, Imperial and University College London; The London School for Hygiene and Tropical Medicine (LSHTM); the LiLaC collaboration between the Universities of Liverpool and Lancaster and Fuse; The Centre for Translational Research in Public Health, a collaboration between Newcastle, Durham, Northumbria, Sunderland and Teesside Universities

Economic Evaluation

iii

ACKNOWLEDGEMENTS

The researchers would like to thank all those who contributed to this evaluation, particularly

the clients of the perpetrator programme, the Foundation for Change staff who facilitated

the study and a large number of other professionals in Doncaster including staff from

Doncaster Domestic Abuse Victim Services, public health practitioners, Doncaster

Metropolitan Borough Council who participated and or facilitated the research. We are also

grateful to the members of the project advisory group and members of the client reference

group who have supported this research over a period of two years.

We are also grateful to research team members from University of Sheffield: Katie Powell,

Mary Crowder, Richard Cooper, Sarah Salway, and Jennifer Burr for their contribution to the

research project. We are also grateful to the public health practitioner participants from

Doncaster council: Debbie Leyden, Susan Hampshaw, Sandra Norburn, Bill Hotchkiss, Cal

Lacey and Lucie Waugh for their help and support throughout the project. Special thanks to

the members of Foundation for Change Programme: Julie Sinclair-Day, Nikeisha Bragger,

Sarah Thompson, Fiona Fischbach and Eleanor Webb.

Economic Evaluation

iv

EXECUTIVE SUMMARY

This report documents the Social Return on Investment analysis (SROI) of a voluntary

domestic violence and abuse (DVA) perpetrator programme offered in Doncaster, England.

The programme was commissioned by Doncaster Council for three years from April 2014 from

an organisation called Foundation (www.foundationdomesticabuse.org). The programme

aimed to deliver specialist support to both men and women aged 16 years and over who

recognise a need to address their abusive behaviour and, therefore, access the programme

voluntarily. Clients were offered personalised and ongoing support through a menu of options

including: an awareness workshop; individual sessions or meetings with support worker; an

eight-week programme of structured group sessions; and flexible drop in hub on completion

of the programme.

Methods:

The SROIA covered an observed period of two and a half years, with benefits forecast for a

further two and a half years (i.e. five-year time horizon). The cost for the Local Authority to

commission the services over this time period was £382,500. The SROIA consisted of six

stages: Identify stakeholders; Map the range of inputs, outputs and outcomes; Identify and

measure key outcomes; Assess attribution, sustainability and displacement; Value outcomes;

Calculate the ratio of attributable benefits to costs and sensitivity analysis.

Findings:

A large number of outcomes could not be measured. However, five outcomes contributed

to calculating the impact of the programme. The following three outcomes provided notable

evidence of impact for the programme.

- Reduced substance abuse reported in a selection of case notes and estimated for all

clients:

o impact on society and

o impact on the individual quality-adjusted life-year

- Improved parenting evidenced by behaviour questionnaire, interviews and reported in

case notes; estimated for all clients.

Economic Evaluation

v

- Reduced time of social workers reported in case notes as information collection and

sharing with social workers.

The following two outcomes, demonstrated small values of impact for the programme:

- Reduced need for accident and emergency health services, assuming that the intervention

resulted in 50% of clients reducing their Accident and Emergency attendances down to

the national average.

- Reduced need for separate housing (fewer incidents of providing refuge for partners),

evidenced from case notes describing improved relationships with co-habiting partners

estimated for all clients.

Benefits were forecast over a five-year period and the SROI ratio was calculated using the

actual costs for the programme: assuming that funding is ceased beyond the 30 months of

the intervention. The SROI ratio was calculated at £2.05, with a net present value of

£401,617.93, and a break-even point during the fourth quarter in year three.

Conclusions:

Future economic evaluations of voluntary domestic violence and abuse perpetrator

programmes should consider the difficulties encountered and reported in this study, and seek

to identify measures and data collection mechanisms for the outcomes that were identified

as important, but could not be measured. Two pervasive problems for evaluations of this type

of programme are the loss of clients to follow-up and difficulties in obtaining outcome

measures for victims and families. Despite the disadvantages of relying on clients’ self-reports

it would be useful to further explore the potential for using proxy measures during the

window of engagement with the service-user. While some measures have been developed

for this study, areas of possible advances are described in detail in this report and in the report

for the full evaluation. The analysis allows clear recommendations, including (a) useful

approaches for those for collecting and measuring outcomes and (b) where further research

may be most valuable.

Economic Evaluation

vi

TABLE OF CONTENT DECLARATION OF INTERESTS ..................................................................................................... ii

ACKNOWLEDGEMENTS ............................................................................................................. iii

EXECUTIVE SUMMARY .............................................................................................................. iv

TABLE OF CONTENT .................................................................................................................. vi

LIST OF TABLES ........................................................................................................................ viii

LIST OF FIGURES ...................................................................................................................... viii

LIST OF ABBREVIATIONS ........................................................................................................... ix

INTRODUCTION .......................................................................................................................... 1

1.1 Local Context ............................................................................................................... 3

1.2 Description of the perpetrator programme ................................................................ 4

1.2.1 Awareness workshop ........................................................................................... 5

1.2.2 Individual sessions ............................................................................................... 5

1.2.3 Eight week structured group sessions ................................................................. 6

1.2.4 Flexible, drop-in support ...................................................................................... 6

METHODS ................................................................................................................................... 8

2.1 Key challenges ............................................................................................................. 8

2.2 Principles and Stages of SROI ...................................................................................... 8

2.2.1 Stage 1: Identifying stakeholders....................................................................... 10

2.2.2 Stage 2: Mapping the full range of inputs, outputs and outcomes ................... 12

Mapping outcomes ...................................................................................................... 12

Identifying and Valuing Input....................................................................................... 12

Details of Financial Input ............................................................................................. 12

Clarifying outputs and outcomes ................................................................................. 14

2.2.3 Stage 3: identification and measurement of key outcome indicators .............. 14

2.2.4 Stage Four: Establishing impact ......................................................................... 17

Economic Evaluation

vii

Deadweight and displacement .................................................................................... 17

Attribution .................................................................................................................... 18

Drop-off ........................................................................................................................ 18

Calculating impact ........................................................................................................ 18

2.2.5 Stage Five: Calculating the SROIA ...................................................................... 19

Projecting into the future ............................................................................................ 19

Calculating the net present value ................................................................................ 19

Calculating the SROIA ratio .......................................................................................... 19

Sensitivity analysis ....................................................................................................... 19

Payback period ............................................................................................................. 20

2.3 Strengths and limitations of the methodology ......................................................... 20

FINDINGS .................................................................................................................................. 22

3.1 Difficulties in measuring outcomes ........................................................................... 29

3.2 SROI Calculations ....................................................................................................... 31

3.2.1 Sensitivity Analysis ............................................................................................. 32

DISCUSSION .............................................................................................................................. 38

RECOMMENDATIONS FOR FUTURE RESEARCH ....................................................................... 40

REFERENCES ............................................................................................................................. 41

Economic Evaluation

viii

LIST OF TABLES

Table 1. 1: Summary of the group sessions ............................................................................... 7

Table 2.1: Principles of SROI ...................................................................................................... 9

Table 2. 2: Details of SROIA related data collection activity ................................................... 10

Table 2.3: Key stakeholders and rationale for inclusion in the SROIA .................................... 11

Table 2.4: Outcomes for stakeholders ..................................................................................... 16

Table 3.1: SROI assumptions and values ................................................................................. 23

Table 3.2: Counted SROI Values ............................................................................................... 33

Table 3.3: 0 to 2.5 year SROI for a 2.5 year funded programme (281 clients over 10 quarters)

.................................................................................................................................................. 34

Table 3. 4: 2.5 to 5 year SROI for a 2.5 year funded programme (281 clients over 10 quarters)

.................................................................................................................................................. 35

Table 3.5: Low sensitivity analysis ........................................................................................... 36

Table 3.6: Sensitivity analysis comparisons ............................................................................. 37

LIST OF FIGURES Figure 2.1: The process of SROIA ............................................................................................... 9

Figure 2. 2: Impact map ........................................................................................................... 13

Economic Evaluation

ix

LIST OF ABBREVIATIONS A & E Accident and Emergency

CAFCAS Child and family court advisory and support service

CRG Client reference group

CSEW Crime survey of England and Wales

DAC Domestic abuse caseworker

DHR Domestic Homicide Reviews

DMBC Doncaster Metropolitan Borough Council

DVA Domestic violence and abuse

F4C Foundation for Change

FGD Focus group discussion

FGM Female genital mutilation

GP General Practitioner

IDVAs Independent domestic abuse advisor

IPV Intimate partner violence

MARAC Multi-agency risk assessment conference

MORM Multifactor offender readiness model

MRC Medical research council

NICE National Institute of Health and Clinical Excellence

NIHR National Institute of Health Research

NSPCC National Society for the Prevention of Cruelty to Children

PAG Project advisory group

PHPES Public Health Practitioners Evaluation Scheme

PHQ-9 Patient Health Questionnaire-9

PSSRU Personal Social Service Research Unit

QALY Quality-adjusted life-year

ScHARR School of Health and Related Research

SD Standard deviation

SMART Sustainable, measurable, achievable, realistic, time-bounded

SPSS Statistical package for social sciences

SROI Social return on investment

SROIA Social Return on Investment Analysis

UCLAN University of Central Lancashire

UK United Kingdom

WEMWBS Warwick Edinburgh Mental Well Being Score

1

INTRODUCTION

This report documents the economic evaluation using a Social Return on Investment Analysis

(SROIA) of a voluntary domestic violence and abuse (DVA) perpetrator programme offered in

Doncaster, England. The evaluation was conducted as part of a wider evaluation programme

funded by the National Institute of Health Research (NIHR) Public Health Practitioners

Evaluation Scheme (PHPES), through the School for Public Health Research (SPHR). The work

was carried out between April 2015 and March 2017, and was led by a team of researchers

from the School of Health and Related Research (ScHARR) and School of Nursing and

Midwifery, University of Sheffield with input from staff from Foundation for Change (F4C),

domestic violence perpetrator programme and Doncaster Metropolitan Borough Council

(DMBC).

Domestic violence and abuse (DVA) remains a major social and public health problem that

intersects social, economic, cultural, geographical and religious boundaries. The World Health

Organization (2014) defines it as a pattern of abusive behaviour perpetrated by an ‘intimate

partner’ or ex-partner resulting in physical, sexual or psychological harm. The acts may include

physical aggression, sexual coercion, psychological abuse and controlling behaviours.

Controlling behaviour refers to ‘a range of acts designed to make a person subordinate and/or

dependent by isolating them from sources of support, exploiting their resources and

capacities for personal gain, depriving them of the means needed for independence,

resistance and escape, and regulating their everyday behaviour’ (UK Home Office, 2013). DVA

has been found to be associated with severe physical and psychological consequences in

victims ranging from substance abuse, depression, posttraumatic stress disorders, suicidal

ideation, injury, and death (Alhabib, Nur, & Jones, 2010; Hamilton, Koehler, & Lösel, 2012).

While development and provision of victims support programmes is necessary, appropriate

programmes aiming at DVA perpetrators to help change their attitudes and behaviours

through understanding the consequences of their abuse are important (Akoensi, Koehler,

Lösel, & Humphreys, 2013; Hamilton et al., 2012).

Economic Evaluation

2

In this report, the term DVA, is used to describe the range of violent or abusive behaviour

between intimate partners or other close family members. Other terms such as ‘Intimate

partner violence’ (IPV), and ‘domestic abuse’ are also used when referring to previous work

using different definitions. The term ‘perpetrator’ is used to refer to an individual committing

DVA and the term ‘victim’ is used to refer to the victim or survivor of DVA.

While some research has been conducted to explore the effectiveness of perpetrator

programmes (Akoensi et al., 2013), methodological issues affecting the quality of studies

make it difficult to derive firm conclusions. Research has also been conducted to determine

the individual and societal cost of DVA (Bonomi, Anderson, Rivara, & Thompson, 2009; Farmer

& Tiefenthaler, 1997; Yodanis, Godenzi, & Stanko, 2000). However, not much information is

available about the cost, cost effectiveness or the value of DVA perpetrator programmes. This

could be due to the difficulties in determining the economic benefits and especially the social

value of such programmes. The value produced by such programmes in the form of changes

in DVA related attitudes and behaviours can be subtle, difficult to measure and realised over

a long time period.

Social Return on investment (SROI) analysis is a framework that enables measurement and

quantification for a broad range of values. The process aims to draw attention to issues of

inequality, environmental degradation and wellbeing by incorporating social, environmental

and economic costs and benefits associated with a particular programme or organisation

(Nicholls, Lawlor, Neitzert, & Goodspeed, 2009; 2012; Wood & Leighton, 2010). In fact, the

Public Services Act that came into force on 31 January 2013, mandates people responsible for

commissioning public services to consider wider social, economic and environmental benefits

of any service prior to procurement processes (Cabinet Office, 2016). The SROI process has

been used to evaluate the impact of a wide range of initiatives including social enterprise

(Rotheroe & Richards, 2007), a transport to work programme (Wright, Nelson, Cooper, &

Murphy, 2009), a community befriending programme (Arvidson, Battye, & Salisbury, 2014)

and peer support for people with dementia (Willis, Semple, & de Waal, 2016). However, very

few attempts have been made to determine the social value of DVA perpetrator programmes

(Nef consultancy, 2013; Perfect Moment, 2010). This report aims to present the findings of a

SROI analysis to quantify the social value of a voluntary DVA perpetrator programme offered

Economic Evaluation

3

in Doncaster, England: to identify and report social outcomes that are not always

measureable with regards to their monetary value.

1.1 Local Context

Doncaster has a relatively high rate of reported DVA and available routine data suggest a

steady increase over the past five years (Doncaster Domestic Abuse Service, 2016). The

reported rate of DVA is higher than other parts of South Yorkshire and is one of the highest

in the country. Over the last year there has been a steady rise in the number of domestic

abuse related crimes recorded by South Yorkshire Police. Forty-seven percent of DVA

offenders were an ex-partner, partner or spouse of the victim. Offenders were

disproportionately male, aged between 20 and 34, of Black ethnicity and unemployed at the

time of the offence, compared with the general population.

Six out of the 11 wards across South Yorkshire with the highest number of DVA offenders are

in Doncaster. The areas in Doncaster with the highest number of reported domestic abuse

offences have high levels of deprivation, and are characterised by: low value housing, low

income, high levels of unemployment, above average levels of mental health issues, drug and

alcohol abuse and high levels of accident and emergency (A &E) admissions, low levels of

educational attainment and high levels of crime and anti-social behaviour (Doncaster

Domestic Abuse Service, 2016). DVA is associated with significant negative health effects as

well as economic costs on victims, their families, and on public services and wider society

(Bellis, Hughes, Perkins, & Bennett, 2012). The overall wider public cost of DVA in all cases for

Doncaster is estimated to be over £110 million (Doncaster Domestic Abuse Service, 2016).

In 2012, Doncaster’s DVA services were subjected to a comprehensive review carried out by

the National Society for the Prevention of Cruelty to Children (NSPCC) and Kafka Brigade. The

resulting strategy set out objectives and key priorities for local agencies and partnerships to

work together for an integrated response taking a whole family approach. The Foundation for

Change (F4C) programme was commissioned as part of the range of related interventions that

included: Domestic Abuse Hub (first response, centre of expertise), prevention and education

(raising public awareness, work with schools/colleges), a Multi-Agency Workforce

Economic Evaluation

4

Development Strategy and Capability Framework, support for victims including housing and

refuge provision and services to support children and young people.

1.2 Description of the perpetrator programme

The perpetrator programme was commissioned by Doncaster Council for two years from April

2014 from an organisation called Foundation (www.foundationdomesticabuse.org) with a

budget of approximately £300,000 and was extended to a further one year till March 2017.

The programme aimed to deliver specialist support to DVA perpetrators, including both men

and women aged 16 years and over. The programme was designed to cater for perpetrators

who recognise a need to address their abusive behaviour and, therefore, access the

programme voluntarily. In common with a number of other DVA interventions for

perpetrators and victims, the perpetrator programme embeds motivational interviewing as a

core element, a directive client-centred counselling style that aims to elicit behaviour change

by helping clients to explore and resolve ambivalence. Every client is supported by a named

support worker who employs motivational interviewing principles (Miller & Rollnick, 2012;

Rollnick & Miller, 1995) and techniques to enhance the client’s sense of self efficacy and

willingness to constructively engage with the programme from the first assessment

appointment onwards.

The programme also uses the trans-theoretical Stages of Change framework (DiClemente &

Prochaska, 1982; Prochaska, 2013; Prochaska & DiClemente, 1986) to sequence

interventions. In addition, the programme recognises the potential for wider contextual

factors to inhibit behaviour change, or act as triggers, and includes one-to-one sessions where

the client and support worker co-produce a support plan, designed to maximise the potential

for behaviour change by addressing barriers to the process for example: financial difficulties,

homelessness, and alcohol abuse. Self-observation and self-monitoring against SMART

(Specific, Measurable, Achievable, Realistic, Time-bounded) targets is encouraged in regular

review sessions.

The programme is delivered jointly by two support workers with additional support from a

programme manager and an administrative officer. Two support workers who were initially

Economic Evaluation

5

involved in the programme have now moved to another programme offered by the

organisation and other two support workers (started work in September 2016) have taken

over the role. Since April 2014 four other support workers and two assistant support workers

joined the programme on short term contracts. All support workers were educated up to

degree level (degree in psychology and criminology). Except two, all support workers (n-10)

who have worked since inception of the programme were females and their age ranged

between 26-41 years (mean 30.4; Mode 28).

Clients are referred by professional organisations using a standard referral form. Clients can

also self-refer in to the programme. Such clients may also be signposted to the perpetrator

programme by various other organisations. Following referral, all clients are assessed for their

eligibility and willingness to complete the programme. Following acceptance onto the

scheme, a client is offered personalised and ongoing support through a menu of options

including: an awareness workshop; individual sessions or meetings with support worker; an

eight-week programme of structured group sessions; and flexible drop in hub on completion

of the programme. In the following section, brief details of these activities are presented.

1.2.1 Awareness workshop

The awareness workshop is the first session clients attend with F4C. These were introduced

to maximise client engagement and reduce the number of dropouts following initial risk

assessment. It is intended to provide appropriate programme information to help prospective

clients understand what will be expected of them and make an informed decision about

involvement in the programme.

1.2.2 Individual sessions

One-to-one sessions or meetings are offered to clients throughout the programme. The aim

of these sessions could be different depending on the individual client needs. The one-to-one

sessions take place at different stages of the perpetrator programme and the examples

include:

- Pre-group work: clients attend two or more one-to-one sessions with their designated

support worker prior to their engagement in group work.

Economic Evaluation

6

- Alongside group sessions; one-to-ones could be requested by clients who require

additional support outside of group sessions or by support workers to address concerns

about client attitudes or commitment to the programme.

- Programme end: final ‘exit’ one-to-one session following completion of the group

sessions.

- As an alternative for clients unable to attend group sessions; for those identified as

unsuitable for group sessions, such as mental health issues which could not be managed

effectively or triggered within the group (i.e. Autism, personality disorder, PTSD) or child

care issues or work commitments making it difficult for people to attend group session.

- One to one sessions were also offered to female perpetrators. Group work sessions for

female perpetrators were also set up at victim services to facilitate attendance.

1.2.3 Eight week structured group sessions

An eight week (three hours per week) series of structured group sessions are offered as part

of the perpetrator programme. In rare cases, the sessions are also offered as one-to-one

sessions as mentioned previously. Table 1.1 provides an overview of the sessions.

1.2.4 Flexible, drop-in support

Clients can continue to access flexible monthly drop-in hubs, beyond the end of the

programme, to ‘touch base’ and talk about any concerns that they may have. This support

arrangement is co-developed with clients who, following completion of the programme, are

indicated as in need of additional support.

Economic Evaluation

7

Table 1. 1: Summary of the group sessions Session Title Session Objectives/Learning Outcomes

What is a healthy relationship?

• To further develop awareness of abusive behaviour in intimate and family relationships

• To practice skills to improve healthier relationships with their intimate partners and families

Managing emotions • To develop an awareness of emotional states that are difficult to cope with • To recognise triggers to our own emotions and develop awareness of

physical signals that accompany emotional states • To recognise unhealthy and healthy ways of coping with difficult mind

states and the affect these have on relationships • Practice skills to improve coping strategies and plan for healthier

relationships

Communication Skills

• To develop an awareness of good communication skills • To practice saying how we feel without being abusive • To practice skills to improve communication with partners and plan for

healthier relationships

Negotiating a compromise

• To practice active listening and assertive statements to resolve conflict without resorting to abuse

• To use a problem solving approach to resolve conflict with partners in a non-abusive manner

• To identify the arguments that most often occur in intimate relationships and plan a healthy way to reach an agreement

The impact of domestic abuse on children

• To understand the effects of DVA on children • To identify what children, need from parents • To plan healthier relationships with children without partners and families

Positive parenting • To improve understanding of the impact of physical/violent discipline methods on children

• To identify healthy ways of encouraging good behaviour and setting up appropriate boundaries

• To weigh up costs and benefits of smacking • To plan ways to be positive parent/role model

Substance misuse • To improve understanding why and how we use alcohol • To understand how alcohol effects relationships • To identify safe limits for alcohol use • To plan for healthier relationships with our partners and families

Relapse prevention Plan

• To develop a detailed plan for improving relationships with partners and families

Note: The objectives are taken from each individual session

Economic Evaluation

8

METHODS

2.1 Key challenges

The approach to economic analysis of the programme was Social Return on Investment

(SROI). “SROI can retrospectively measure outcomes that have already occurred (evaluative-

type) or can prospectively predict how much value will be generated if the intervention meets

its intended outcomes (forecast-type)” (Banke-Thomas, Madaj, Charles, & van den Broek,

2015, p. 1471). However, there are considerable barriers to empirically assessing the impact

of DVA perpetrator programmes in terms of valuing outcomes. Some of the main difficulties

include:

Low levels of voluntary participation in evaluation

Loss of participants to follow-up

Difficulty accessing data, which could quantify the extent of the benefits experienced

by the victims, and any children

Difficulties accessing potential beneficiaries

The breadth of benefits being uncertain; does it affect subsequent partners, the

wider community (e.g. police, health, employer, welfare, school etc.)

Potentially very long time-periods for full realisation of benefits, especially effects on

children into adulthood (e.g. PTSD, depression, replication of learned behaviour)

Therefore, some potentially important outcomes were not included in the final calculations

as it was not possible to collect data regarding all impacts of the programme (see Section 3.1

which sets out the difficulties in measuring outcomes).

2.2 Principles and Stages of SROI

For the purposes of this study an evaluative type social return on investment analysis (SROIA)

was carried out. The SROIA, in line with Department of Health’s best practice guidance (DH,

2011), is based on seven principles as suggested by the SROI network and summarised in table

2.1. The SROIA involves six stages as illustrated in Figure 2.1. The following section of the

report describes each of the five stages, prior to reporting of the SROI analysis that were

undertaken for this work.

Economic Evaluation

9

Table 2.1: Principles of SROI

Principal Definition

Involve stakeholders To inform what gets measured and how this is measured and valued by

involving stakeholders

Understand what

changes

Articulate how change is created and evaluate this through evidence

gathered, recognising positive and negative changes as well as those that are

intended or unintended

Value the things that

matter

Use financial proxies in order that the value of the outcomes can be

recognised. Many outcomes are not traded in markets and as a result their

value is not recognised

Only include what is

material

Determine what information and evidence must be included in the accounts

to give a true and fair picture, such that stakeholders can draw reasonable

conclusions about impact

Do not over-claim Only claim the value that organisations are responsible for creating

Be transparent Demonstrate the basis on which the analysis may be considered accurate and

honest, and show that it will be reported to and discussed with stakeholders

Verify the result Ensure independent appropriate assessment

Figure 2.1: The process of SROIA

Stage One

•The identification of key stakeholders

Stage Two

•Mapping of the full range of inputs, outputs & outcomes

Stage Three

•Identification and measurement of key outcome indicators

Stage Four

•Establishing Impact: Assessment of attribution, sustainability and displacement of other services

Stage Five

•Calculating the ratio of attributable benefits to costs plus sensitivity analysis

Stage Six

•Reporting using and embedding

Economic Evaluation

10

2.2.1 Stage 1: Identifying stakeholders

After establishing the preliminary scope of the project, the first stage of SROIA involved

identifying stakeholders to be included in the exercise. An initial group of stakeholders2

including service providers, commissioners and academics were consulted to identify further

stakeholder groups and potential individuals to contact. Following this, we conducted three

individual interviews and three SROIA workshops (each covering all three stages of the SROIA)

involving nineteen individuals representing various stakeholder groups (Table 2.2). The

participants included staff members of the DVA perpetrator programme, representatives of

associated services including domestic violence victim services, commissioners, public health

practitioners, social workers, clients of the perpetrator programme and partners of the clients

(victims/ survivor of violence). Further information about data collection is presented in the

next section.

Table 2. 2: Details of SROIA related data collection activity

Workshop 1 Workshop 2 Workshop 3

Individual Interviews

Number of participants

4 5 10 3

Age (range)

25-44 25-44 32-54 25-40

Gender Male Female

2 2

0 4

1 9

0 3

Description of participants

Two domestic abuse case workers (DACs), and two victims whose partners were undertaking perpetrator programme

Service users of perpetrator programme

DVA practitioners from perpetrator programme and victim services; public health professionals; commissioners; researcher

2 Stakeholders refer to any group of individuals or organisation that is affected by the

service/intervention and experience.

Economic Evaluation

11

Through various interactive exercises, the participants identified other key stakeholders, their

contribution and impact on the programme and vice versa. Table 2.3 outlines the key

stakeholders included and the rationale for inclusion and exclusion in the SROIA.

Table 2.3: Key stakeholders and rationale for inclusion in the SROIA

Stakeholder/Role Reason for inclusion in SROIA

Commissioners Local Council, public health, South Yorkshire Police

Provide finances for the programme. Commission various services in the town with an aim to reduce incidence and prevalence of DVA

DVA Perpetrator Programme Staff

Deliver DVA perpetrator programme and ensure that it meet the needs of service users and commissioners

Service users (perpetrators) Access DVA perpetrator programme in an effort to change their behaviour. Involvement in the programme can help them improve their behaviour and consequently improve their relationships with their partner and others

Partners of service users Are affected by the programme activity as improvement in the abusive behaviour of their partner can improve their intimate relationship

Children of service users and their partners

May be affected by the DVA in their family. Improvement in the intimate relationship of their parents/ guardians can have positive impact on their life

Immediate family and friends May include extended family of perpetrator/ service user and their partner. They may act as support system for the victim, children and/or the perpetrator.

Children services Indirect beneficiary: Aim to keep children safe and may have to intervene in situations where children are living in DVA affected families.

Social services Indirect beneficiaries: Aim to help children, adults and families to take control of and to improve their lives in conditions where their security, safety or ability to participate in civic life are restricted.

DVA victim support services Indirect beneficiary: Aim to support victims of DVA and keep them safe

Police and criminal justice system

Indirect beneficiary: Aims to reduce the number of DVA incidents in the community and react in the event of an incident

Health services Indirect beneficiary: Benefit from short-term reduction in treatment for physical harm and longer-term mental health and alcohol and drug abuse

Stakeholders Reason for Exclusion

School and Education system May not be aware of involvement of perpetrator in the perpetrator programme and therefore may not be able to comment about the programme.

Neighbours Proportion of affected neighbours will be low and difficult to identify and measure impact.

Economic Evaluation

12

Neighbourhood teams May not be aware of the perpetrator involvement in the programme; Difficult to identify and measure impact.

2.2.2 Stage 2: Mapping the full range of inputs, outputs and outcomes

Mapping outcomes

An impact map sets out the programme theory of the intervention, and communicates how

an intervention is expected to make a difference, and how inputs are expected to result in

outputs which lead to outcomes. An SROIA impact map is similar to a ‘theory of change’ or

‘logic model’; it describes the overall rationale for the programme and sets out linked

assumptions of the relationships between activities and outcomes. Activities mentioned

above played a key role in mapping inputs and outputs and identification of key outcome

indicators and appropriate data sources to measure those outcomes. Figure 2.2 provides a

visual overview of the Impact Map. Please note that the stages outlined in the Impact Map

correspond to stages 1-3 outlined in Figure 2.1.

Identifying and Valuing Input

Inputs are things that stakeholders contribute in order to make activities possible. The inputs

were relatively straightforward to identify with the help of the perpetrator programme staff.

For example, the financial input was primarily from the funding received from the

commissioning body. This funding was then used to pay for the staff salaries, running cost of

the office premises etc. Contributions from clients and other stakeholders were mainly in the

form of their time, which is considered as an input, however, according to standard SROIA

approach, “…the time spent by the beneficiaries on a programme is not given a financial

value” (Nicholls et al, 2009, p.32). Consultations with stakeholders established that inputs

from other stakeholders did not determine whether the programme activities went ahead.

The only exception to this was the time spent for referral and signposting from other

organisations, though, this was considered to be part of the everyday business of these

stakeholders and not an additional activity. Therefore, the time of practitioners in other

organisations (e.g. social worker, probation, and children’s services) has not been costed.

Details of Financial Input The original contract for the service was paid for by the Local Authority at a rate of £306,000

for two years, which was then extended for a further six months. Costs were scaled up to

Economic Evaluation

13

Figure 2. 2: Impact map

Valu

e Ou

tcom

e

Better family relationship

Reduction in DVA perpetration

Improved physical & mental health

Improved parenting: outcomes for children

Reduced housing needs

Reduced demand on other agencies

Improved employability

Reduced substance abuse

Measu

re Ou

tpu

t

Increased awareness of personal behavior and its impact leading to changes in DVA attitudes and behavior

Increased awareness of available resources

Numbers of clients completing aspects of the course

Assistance with wider contextual issues (e.g. housing)

Communication and coordination with other agencies

Activity

One-to-one support session

Eight group sessions

Drop in Hub

Personalized support

Increased interaction opportunities

Opportunities to reflect & role modeling

Qu

antify In

pu

t

Staff

Facilities

Resources

Stakeho

lders

Commissioners

Staff

Service users & partners

Children

Family and friends

Children services

Social services

DVA support service

Police and criminal justice system

Health services

Economic Evaluation

14

two and a half years, resulting in a total cost of £382,500. The service was provided in

quarterly cycles (four per year) as an eight-week course, with four to five weeks between

courses. Costs and values of returns were calculated for each cohort on a quarterly basis; the

study period therefore covered 10 quarters/cycles, each with a cost of £38,250.

Clarifying outputs and outcomes

Outputs are quantitative summaries of activities. Sometimes outputs are repeated for several

stakeholders at this stage because they form part of the theory of change. However, repeated

outputs are not counted in the final calculation, so double counting cannot occur (Nicholls et

al., 2012). As mentioned earlier, the data collection activities mentioned above not only

helped identify key stakeholders but also helped in mapping inputs and outputs and

identification of key outcome indicators and appropriate data sources to measure those

outcomes.

2.2.3 Stage 3: identification and measurement of key outcome indicators

The perspectives of the different stakeholders are at the centre of the valuation process,

particularly for guiding the identification and measurement of key outcome indicators. We

used a mixed-method, theory-driven evaluation approach involving a wide range of

stakeholders, including service commissioners, service providers, clients, social workers and

other professionals (Nicholls et al., 2012). In addition to the SROIA specific data collection

activities mentioned above, we used various other strategies to collect empirical evidence

(for the larger evaluation of the programme3) and these are summarised below (Ali, Ariss,

Powell, Crowder, & Cooper, 2017).

20 in-depth interviews including nine open-ended individual interviews and eleven

repeat narrative interviews involving a total of 14 different clients.

Five iterative, ‘theory-focused’ interviews involving six members of staff of the

perpetrator programme.

3 An electronic copy of the full evaluation report is available at: https://www.sheffield.ac.uk/snm/research/doncaster

Economic Evaluation

15

11 in-depth interviews involving 12 stakeholders including commissioners, social

workers, professionals from other services such as the police, probation, women’s

centre, domestic abuse victim services, and children's services.

Three focus group discussions (including one with perpetrator programme staff and

two with other stakeholders) involving 21 participants.

Ethnographic observations of six one-to-one sessions between clients and their

keyworkers. We also reviewed recordings of each of the eight group sessions.

Case note review of 33 clients

Review of routinely collected data of 281 clients.

Pre and post-programme attitude survey completed by 42 clients.

Self-administered questionnaire administered at the start or the programme (T0;

N=42), at the end of the programme (T1; N=11)) and three months’ post completion

of the programme (T2; N=4).

Final calculations were based on empirical evidence. Outcomes for which we could not

identify appropriate evidence were included in the SROIA, where necessary, but given a value

of zero to allow values to be inserted in the model, if they become available. Outcomes for

which no evidence was available to support them were not included in the sensitivity analysis;

in line with the SROIA principles of only including ‘what is material’ and being careful not to

over-claim.

All assumptions in the SROIA model are based on the most appropriate evidence, together

with expert judgement. Following the ethos of SROIA, these are conservative and transparent,

such that these are open to challenge so that they are either improved or displaced in time

by empirical evidence. A key assumption necessary to conduct this SROIA is that; where data

availability is limited, and evidence exists for a proportion of a representative sample of

service-users, then this is assumed to pertain to all clients at the same ratio across the

programme.

Data collected from various sources (individual interviews, focus group discussions, SROIA

stakeholder activities, and questionnaires) helped identify a wide range of changes for various

Economic Evaluation

16

direct and indirect beneficiaries which might include; clients / service users, partners of

service users (victims), children, family members and friends, other services (local services,

social services, children services, police/ criminal justice system, health services, homeless

charities/ refuges and Department of work and pensions and employers) as shown in Table

2.4.

Table 2.4: Outcomes for stakeholders

Stakeholders Outcomes (based on how stakeholders describe the change)

Perpetrators - Improved parenting skills

- Better mental health/well-being

- Lower use of substances/alcohol

- Make more informed decisions

- Increased motivation to change

Victims - Reduced incidents of DVA and repeat incidents

- Reducing time that victims spend in abusive relationship

- Improved family relationship

- Improved health and well being

Immediate family and friends

- Fewer incidents of providing refuge for partners

- Lower levels of worry/ Improved family relationships

Children - Fewer children involved in incidents of DVA

- Improved school attendance

- Improved well-being/mental health

Victim services (DACs, Counsellors, refuge staff) and other agencies

- Increased number of referrals to victim services

- Changes in type of support requested (less crisis)

- Preventing escalation to MARAC

Social Services/ Local Authority Children's services

- Reduced costs of supporting/caring for children and family members

o Reduced time of Social Worker (e.g. Children's Services)

o Reduced number of children requiring adoption

Police/ vulnerability units/ criminal justice system

- Reduction in incidents of abusive behaviour reported to police or other agency (Reduced numbers of police call outs/policing cost reduced rate of domestic abuse)

Employers, Department of Work and Pensions

- Attendance of employee/perpetrator/ victims at work; performance of perpetrator at work

Health services - Reduced health care needs (DVA related)

Homeless charities, women’s refuges Housing services

- Reduced homelessness/ need for separate housing

Economic Evaluation

17

For the later stages of the SROIA, we counted values that had been attributed to outcomes,

to arrive at the impact of the programme. Where evidence of outcomes was not available,

these have been given zero occurrences so that they do not contribute to the final calculation.

2.2.4 Stage Four: Establishing impact

Establishing impact refers to determining how much of the outcome would have happened

anyway in the absence of the programme/ intervention. It also explores what proportion of

the outcome can be isolated as being added by the programme activities (Cabinet Office,

2016; Nicholls et al., 2012). Establishing impact is important as it reduces the risk of over-

claiming. Measures and values attributed to outcomes are detailed in the findings section

(Table 3.1)

Deadweight and displacement

Deadweight is a measure of the amount of the outcome that would have happened anyway

without the programme. This was both assessed by stakeholders through recourse to their

expertise and experiences and established through available evidence. Cabinet office

guidance indicates that this is rarely a perfect estimate: “measuring deadweight will always

be an estimate since a perfect comparison is not possible. Instead, you need to seek out

information that is as close to your population as possible. The more similar the comparison

group, the better the estimate will be” (Nicholls et al, 2009, p.56). It is measured as a

percentage, which is deducted from the total quantity of the outcome. To understand this

better, it may be useful to consider an example. For instance, if 30% more people on a

programme gain employment than would have been expected without the programme the

deadweight is calculated as 100% (business as usual) expressed as a percentage of 130%

(programme outcome) = 77% deadweight.

Displacement is an assessment of the extent to which the outcome measures have

demonstrated a change, which has been absorbed elsewhere. For instance, reductions in DVA

could change into other forms of violence to others. This would require an adjustment by

either including the stakeholder now being affected or estimating the effect of double

Economic Evaluation

18

counting as a percentage to be deducted from the total. No displacement was calculated for

the outcomes included in the model.

Attribution

Attribution is an assessment of the extent to which outcomes were caused by other external

influences (e.g. other programmes, organisations or people). The extent to which outcomes

are assessed at being attributable to other sources is deducted from the programme outputs.

“It will never be possible to get a completely accurate assessment of attribution. This stage is

more about being aware that your activity may not be the only one contributing to the change

observed than getting an exact calculation. It is about checking that you have included all the

relevant stakeholders.” (Nicholls et al, 2009, p.56).

Within Doncaster, there are no similar voluntary interventions aimed at addressing the

abusive behaviour of low-risk DVA perpetrators. There are other services that clients can

access to potentially address problematic behaviour (such as substance abuse), which is also

addressed by elements of the programme. However, evidence was not available for clients

accessing other services. Indeed, if clients were considered to need external interventions,

they were not admitted onto the programme.

Drop-off

Drop-off is concerned with how long the outcomes last. The amount that the value of an

outcome will be expected to reduce over time, or be influenced by other factors, so that

attribution becomes lower. It is calculated for outcomes that last longer than one year, as a

fixed percentage that is deducted year on year. This is calculated on a case-by-case basis for

each outcome. Whilst the full benefits of some outcomes are only realised over time, others

are short-term and rapidly decrease over time. Details of these calculations are included in

the findings section (Table 3.1). The time horizon for the analysis is a five-year period.

Calculating impact

Impact is calculated by multiplying the financial proxy by the quantity of an outcome to give

a total. From this total, percentages are then deducted for deadweight, displacement and

Economic Evaluation

19

attribution. This is repeated for each outcome to give the impact for each. These individual

impacts are then combined to give a figure for the total impact.

2.2.5 Stage Five: Calculating the SROIA

Projecting into the future

Projecting the value of outcomes into the future consists of setting out the value for each

outcome for a determined time-period (i.e. five years), and copying across the number of

time periods it will last. Any drop off calculated above is then deducted. Details of these

calculations are included in the findings section (Tables 3.3 & 3.4)

Calculating the net present value

The Net Present Value (NPV) is calculated by comparing costs and benefits over a time period.

Discounting of 3.5% per annum over the projected time periods is used to account for money

today having greater value than money in the future (For the public sector, the basic rate

recommended in HM Treasury’s Green Book (2016) is 3.5%). This gives the present value of

benefits. The value of investments is then deducted to calculate the NPV.

Calculating the SROIA ratio

The SROI ratio is calculated by dividing the present value by the value of inputs. The SROI ratio

was calculated using two approaches to valuing the inputs; one of which reflects the actual,

short-term funding for the programme, and one approach which demonstrates the return for

a hypothetical longer-term funding model. The first of these used the actual costs for the

programme, taking account of the fact that the funding for the programme ceased. The

second approach assumed that the funding for the programme would continue over the same

five-year period as the benefit calculations.

Sensitivity analysis

After calculating the ratio, it is important to assess the extent to which this would be changed

by altering some of the assumptions from previous stages. The sensitivity analysis assesses

the importance of various elements of the model. It is therefore possible to explore which

assumptions have the greatest effect on the model. The standard requirement is to check

Economic Evaluation

20

changes to: estimates of deadweight, attribution and drop-off; financial proxies; the quantity

of the outcome; and the value of inputs, (where non-financial inputs have been valued).

To estimate the sensitivity, calculations were made to explore the effect that would result

from changing each estimate to lower-level assumptions, both individually and for the whole

model. By calculating this, the sensitivity of the analysis to changes in estimates can is shown.

This reporting of the model including lower-level assumptions demonstrates the amount of

change to the model necessary to make the ratio change from positive to negative or vice

versa (i.e. lower than £1 value for £1 investment)..

Payback period

The ‘payback period’ describes how long it would take for the investment to be paid off in

terms of the accrual of social value. Specifically, it describes at what point in time the value

of the social returns starts to exceed the investment. This can be used to determine risk in a

project. While a short payback period may seem less risky, a longer payback period is often a

feature of activities that are able to demonstrate significant long-term outcomes. For these

types of interventions longer-term core funding is required.

2.3 Strengths and limitations of the methodology

As with any approach, an SROIA approach has a number of strengths and limitations and it is

important to appreciate these early on in the process. In terms of strengths, the SROIA

enables identification and presentation of a single monetised ratio of positive and negative

outcomes that can be easily understood and conveyed to a variety of stakeholders. The SROIA

values the engagement of stakeholders and therefore provides a platform where

stakeholders are involved in not only informing the outcomes but the values and

meaningfulness of those outcomes from stakeholders’ perspectives. This facilitates

transparency and accountability. An additional strength is the ability to value more than

simple organisational or individual fiscal transactions; placing value on longer-term social

benefits, which are often the most important intended outcomes of this type of social

intervention.

Economic Evaluation

21

Like any quantitative approach, one of the biggest limitations of the SROIA, however, relates

to difficulties associated with assigning financial values to soft outcomes. Another limitation

includes difficulties in determining the counterfactual (what would have happened without

the investment), although ‘deadweight’ calculations are designed to account for this. It is also

important to remember that monetary values presented as a result of SROI analysis (for every

£1 of investment, a value of £X is returned) is not an actual value. It simply represents a proxy

of an organisation's activity and assumptions regarding holistic value for money rather than

simply changes in financial flows or the generation or cash saving, which can be

communicated in a clear and concise way. This approach is also sensitive to assumptions

made about the validity of outcomes, value of these outcomes and the number of specific

outcomes to be counted. Notwithstanding this, SROIA is a recognised and valued impact

reporting method, which puts stakeholders at the heart of the evaluation.

Economic Evaluation

22

FINDINGS

As discussed earlier, a wide range of stakeholders were consulted to identify appropriate

outcomes to be included in the SROI model. In order to adhere to the rule of SROI to ‘value

the things that matter’, all of these were considered for inclusion. All identified outcomes are

listed below in Table 3.1. Outcomes for which no evidence was identified (i.e. no recorded

instances) are all included in the model, but have no effect on the calculations as they are

indicated by a ‘Quantity of Outcome’ as £0.00. Below table 3.1 there is a list of these outcomes

for which there was no evidence and reasons why.

It is important to note that the large number of potential outcomes with no attributed values

is mostly due to not having adequate measurement processes to capture instances of these

outcomes, rather than the evaluation demonstrating no impact on these outcomes.

Quantification of these outcomes could be included in the model in future, if improvements

are made in outcome capture.

The key outcome themes for which there was evidence identified during the mixed-method

evaluation of the programme are:

- Reduced substance abuse:

o impact on society and

o impact on the individual quality-adjusted life-year (QALYs)

- Improved parenting

- Reduced time of social workers

Minor outcomes, for which there was some evidence of small benefits are:

- Reduced need for accident and emergency health services

- Reduced need for separate housing: Fewer incidents of providing refuge for partners

Table 3.1 details all of the assumptions, values and recorded instances of outcomes. Following

this, the text describes details of all outcomes; including reasons why they were not included

in the final calculations.

Economic Evaluation

23

Table 3.1: SROI assumptions and values

Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter

Quantity of Outcome

Total Value

Reduction in DVA and repeat incidents: physical health

1 x A&E attendance 2014/15= £132 per incident No evidence available on whether the programme prevents A&E attendance of partners or not. Deadweight has not been calculated, but should take into account comparable A&E attendance rates.

£132.00 0.00 £0.00

Reduced need for victim support: The average cost of giving support to each victim-survivor by an independent advocacy domestic violence scheme = £501 per incident

No evidence available on whether the programme reduces the need for victim support or not. Number of victims receiving support is not known. Deadweight should consider victims no longer needing support due to termination of relationship/imprisonment

£501.00 0.00 £0.00

Reduction in DVA and repeat incidents: mental health

Women suffering DVA used four times as many mental health resources as average. If per capita cost of mental health services are multiplied by 4, this generates a cost of £332 per abused woman.

No evidence available on whether the programme results in reduction of mental health services used by partners

£332.00 0.00 £0.00

Reduced accident and injury

Reduced A&E attendance of client 1 x A&E attendance 2014/15= £132.

A&E attendance of cohort (55 cases=20%) compared to national average 29-65 year olds (13%). There was little evidence to suggest that there is a reduction in A&E attendance as a result of the intervention. However, a reduction in violent behaviour and alcohol consumption was reported, and the clients had a slightly (7%) higher incidence of attendance than the national average. It could be claimed as a reasonable assumption that as a result of the intervention 50% of clients reduce A&E attendances to the national average: 50% of difference would=2 cases, or 0.2 per quarter. As there is low confidence around this assumption, reference should be made to the effect of not including this assumption of impact as shown in the sensitivity analysis.

£132.00 0.20 £26.40

Economic Evaluation

24

Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter

Quantity of Outcome

Total Value

Improved mental health

Completing a screening questionnaire (£33), followed by care management for those found to be suffering from, or at risk of developing, depression and/or anxiety disorders (£254 for 6x face to face CBT). Not included in calculations as this is an output not outcome. Evidence of improved mental health is required

Screening only is conducted with health questionnaire. 10% deadweight assumes that 1 in 10 clients receive alternative mental health screening within health services. 5% per quarter drop off estimates that repeat screening might be required.

£33.00 0.00 £0.00

Reduced need to provide refuge for victims

Family and friends who help with accommodation estimated at the same rate as Housing Benefit for 28 weeks = £1,509 for each woman and her family. Per quarter =£647. The 2001 BCS IPV found that 39% of women moved out, at least temporarily, after the incident of domestic violence that they defined as the worst. Of these 90% went to stay with family and friends, while 4% went to a refuge. The accuracy of assumptions for this benefit would be improved with evidence to indicate how long a victim might spend in alternative accommodation; this would allow a more informed drop-off value to be applied

Not many clients were co-habiting on access to the programme. No evidence is available on whether there was an increase in co-habitation. However, 5 clients specifically described [in routine data] improved relationships with partners. The assumption is that, for the partners of these clients, this has prevented escalation of DVA to the point that they feel the need to move out. Drop off at 50% per quarter allows for the temporary nature of living in alternative accommodation. It is assumed that on average half of all partners would return home every 13 weeks anyway, without the intervention (<1% benefit after 2 years).

£647.00 0.50 £323.50

Average non-London Refuge rent/week = £416.29: 1 month x4= £1,665.16

Few clients were co-habiting on access to the programme. No evidence of increase in co-habitation. Very small percentage of women leave home to stay in a refuge.

£1,665.16 0.00 £0.00

Reduced substance abuse

The average harmful drinker costs society £2,500 annually; the average Class A drug taker costs society £13,200 annually in the UK (2009 prices). Alcohol-related problems, crime and anti-social behaviour currently cost society over £25 billion a year. This financial proxy was used in a New Philanthropy Capital case study of an O2 Think Big grant given to a youth club in Carlisle.

Case notes report recognising problem and subsequent stopping or moderating drinking. Drop off at 10% to account for relapse rates (Finney & Moos, 1991) Search of 31 case notes for high number of references for 'Alcohol' (x 4 cases=13%: 41, 57, 35 & 14) revealed a number of ‘FIT Kit’ one-one sessions for alcohol abuse (6, 8, 4 & 3) average number = 5.25 sessions.

£738.27 3.65 £2,281.25

Economic Evaluation

25

Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter

Quantity of Outcome

Total Value

Costs have been inflated to 2009 prices (from the baseline of 2003 figures). These costs apply across central and local government, not just the NHS Valuation Source: Cabinet Office and PM's Strategy Unit (2003) Alcohol misuse: how much does it cost? Report Source: NPC O2 Think Big case study: Danny Boyle's Youth Centre Reference: Value £2500 per harmful drinker per year (£625 per quarter, adjusted for inflation to 2017= £738.27). 100% attribution and 10% drop off. (http://www.globalvalueexchange.org/ valuations/8279e41d9e5e0bd8499f2bed)

Assume this is representative of cohort (13% of 281)= 36.5 cases total over programme = 3.65/quarter.

Reduced substance abuse

Universal screening by GPs of all patients, followed by a 5-minute advice session for those who screen positive for alcohol misuse = £18.40 [28.1 clients avg per quarter]; This valuation of a similar service is included for information only as this is an output rather than an outcome.

The programme included screening followed by intensive group session on substance abuse rather than a 5-minute advice session as defined in the valued intervention. So costings are very conservative in terms of the intervention. Zero Deadweight: evidence shows that screening in primary care is at a very low level. 5% per quarter drop off estimates that screening will be required at least every 5 years on average

£18.40 0.00 £0.00

Reduced substance abuse

Prevention of harmful drinking: quality- adjusted life years. FIT Kit 1:1 alcohol sessions. The state of the nation facts and figures on England and alcohol (alcoholconcern.org.uk/assets/files/PressAndMedia/state.of.the.nation.pdf.) 12.1 million harmful drinkers £21.9 billion in human costs (reduced quality- adjusted life years). Value £1809.91 per person per year (/4)= £452/quarter

Text search of 31 case notes for high number of references for 'Alcohol' (x 4 cases=13%: 41, 57, 35 & 14) revealed a number of FIT Kit one-one sessions for alcohol abuse (6, 8, 4 & 3) average number = 5.25 sessions. Assume this is representative of cohort (13% of 281)= 36.5 cases total over programme = 3.65/quarter. Case notes report recognising problem and subsequent stopping or moderating drinking. Drop off at 10% to account for relapse rates (Finney & Moos, 1991)

£452.00 3.65 £1,649.80

Reduced demand on

F4C staff members assist social workers: Time per hour of Children's Services Social Worker @ £54 per hour of client-related work (5x£79=£395 including

Case note reviews demonstrated close working with social workers and sharing of detailed information, by email and phone calls @ 5 Hours contact per quarter

£270.00 10.30 £2,781.00

Economic Evaluation

26

Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter

Quantity of Outcome

Total Value

other agencies: Children

cost of qualifications). 5x£54=£270. 103 total referred by children's services or self-referral via Children's services

per client (4x1 hour one-to-one plus conservative estimate of information gathering from the group sessions at 1 hour per client from a total of 24 hours). Total 5 hours. Deadweight is zero, as intensive access to voluntary perpetrators is not managed by other agencies. Attribution is 100%. Drop off is zero as this is an ongoing and renewed process for each client

Outcomes for children: Improved parenting

Fiscal savings to public sector agencies derived from the delivery of parenting programmes for parents of five-year old children with conduct disorder -; the research assumes that, for parents completing programmes, 33% of children improve to 'no problems' and 5% to 'moderate conduct problems', but behaviour changes are not sustained beyond one year for 50% of children who initially improve. Attribution is 100%, as all outcomes are attributable to the programme. Deadweight is zero as it is assumed that clients do not access similar services, drop off is calculated at 0, as it is built in to the model. http://www.globalvalueexchange.org/valuations/584580a71c584a1f40e4384

Programme evidence: Behaviour questionnaire ‘The children have not been affected by the abuse’ T0=(17) 40.5, T1=0 (0), T2=04 (100). These results support the programme theory that increased awareness of the effect of their behaviour (T0-T1 children reported as being affected by DVA 59.5%-100%), results in changed behaviour (T2=0%). Limitations of this evidence are that small numbers completed T2 questionnaires and the analysis of responses was not linked to individuals. However, this is also supported by the qualitative findings regarding the reported impact of the session on the effects of DVA on children and corresponding reports of changes in behaviour to protect children from the harms of DVA. Age 6 (savings over the first year following delivery of the parenting programme), £324; age 7-16 (savings for years 2-11 after programme delivery), £2,493 (averaging out at £249 per year over the ten year period); age 17-30 (savings for years 12-25 after programme delivery), £551 (£39 per year over the 14 year period). The outcome is therefore valued at £324 per year for 4 quarters + £249 per year for 6 quarters (£373.50) (697.50/10=£69.75 per quarter). Only applicable to completed clients with children n= 66 (6.6 per quarter).

£69.75 6.60 £460.35

Economic Evaluation

27

Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter

Quantity of Outcome

Total Value

Outcomes for children: Mental health

Mental Health Foundation (2005) report one pilot study, of children aged 4-8 referred with conduct disorder, found that the mean extra cost was £15,282 a year (range £5,411-£40,896).

No current evidence available on whether or not the programme prevents children developing MH problems. Possible external evidence of incidences of CAMH problems in abusive households and effectiveness of similar interventions. Zero deadweight and 100% attribution should be applied if only prevented incidences resulting from the programme are counted

£3,820.50 0.00 £0.00

Outcomes for children: stress management

Improved relationships with children and reduced exposure to DVA, assumes that stress management services will not be required. Stress management course for young person £630 (UK council for psychotherapy). This value is not included in calculations due to the possibility of double counting with benefits for improved parenting

Clients specifically reported better relationships with children and better understanding of the effects of DVA on children [3 cases from routine data]. However, as this is not specific routinely collected data, more cases are possible. Deadweight is zero, as there is no evidence that the clients would seek or be able to access similar behavioural change and educational interventions. Self-reported attribution is 100%. Drop off at 10% per quarter is an estimation of the likelihood that clients will use the knowledge they gain less as time goes by and they forget the learning.

£630.00 0.00 £0.00

Improved life decisions

Make more informed decisions and more motivated to change. Life coach (minimum for 10 weekly sessions) = £600. completers per quarter=12.1. This valuation is included for information only. It is not included in calculations as provision of life-coaching is an output not outcome. Despite this being considered an important outcome by stakeholders, it was not possible within the scope of this study to adequately define and measure ‘improved life decisions’.

Motivational interviewing and other behaviour change techniques embedded in sessions over 12 weeks: completers only. Deadweight is zero, as there is no evidence that the clients would seek or be able to access similar behavioural change and educational interventions. Drop off at 10% per quarter is an estimation of the likelihood that clients will use the knowledge they gain less as time goes by and they forget the learning. Attribution = 60% (i.e. 6 sessions) to prevent double counting of parenting sessions, valued above.

£600.00 0.00 £0.00

Improved employability

Assume savings for the state are equivalent to the amount a family can earn on benefits in a year £500/

No evidence available on changes in employability. £122,200.00 0.00 £0.00

Economic Evaluation

28

Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter

Quantity of Outcome

Total Value

(employment and attendance at work)

couple and single parent and £350/single per week for one year: value £122,200

F4C clients had a lower rate of employment than the general Doncaster population (44.1% compared to 72.5%); more clients were looking for work (34.5% compared to 26.4%). However, there were much lower numbers (1.8%) than the general population receiving disability allowance (1.2%) or long-term sick (6.5%). This is probably due to the relatively low age of the F4C clients.

Social Work & Adoption Foster services

Adoption: cost for placement and ongoing support = 1 child=£27,000; 2 children = 150%; 3 or more children= 200% Plus £750/month ongoing.

We were not able to access data regarding any reduced need for adoption or statutory child care. Deadweight and attribution (estimated at 50%) will depend on quality of the evidence and causal links of outcomes with the programme.

£27,000.00 0.00 £0.00

Fostering £603 per child per week No evidence available to indicate whether the programme reduced the need for adoption or statutory child care

£603.00 0.00 £0.00

Local authority care £3,181.60 per resident week No evidence available to indicate whether the programme reduced the need for adoption or statutory child care

£3,181.60 0.00 £0.00

Criminal justice system: Police call outs

Assuming reduced call outs @ common assault £215; Other £1,000; serious wounding £9,127; sexual offences £3,837

No evidence available to indicate whether the programme resulted in reduced call outs. However 55/275 previous arrests or convictions for assault (and assault with other crime) in previous 6 months

£215.00 0.00 £0.00

Educational services

SROI study of troubled families attributed costs for Education Welfare Officer of £1188

Some evidence from case note review of reduction in school absenteeism from between 60%-100%. However, this evidence was not included in calculations as it was not reliably or systematically collected. And deadweight will be high because of involvement of other agencies e.g. The child has as a keyworker/counsellor who she can see every morning

£1,188.00 0.00 £0.00

Total £8,605.10

Economic Evaluation

29

3.1 Difficulties in measuring outcomes

A brief summary of difficulties in measuring instances of outcomes is given below. Ways to

address these should be considered as recommendations for evaluating SROIA for

perpetrator programmes in future.

Better mental health/well-being: Whilst validated mental health assessment tools are used

(WEMWBS, PHQ-9 & GADS), these are only completed at one point during the intervention.

If these were completed pre and post completion of the course, a value could be placed on

any identified change.

Make more informed decisions: This is a rather vague concept, which was not consistently

articulated by stakeholders during the evaluation, and could be applied to a very wide range

of decisions. It would be beneficial to explore areas where clients are making poor decisions,

and aspects of the course that are aimed at targeting these decision making processes. To a

large extent this process has been applied in the case of decision making around the use of

alcohol and drugs. However, there is scope to apply this in other areas, and then use measures

that target specific outcomes.

Increased motivation to change: a particular problem with measuring this outcome is that

motivation to change is an important part of the initial assessment process. Clients are

therefore encouraged to display high levels of motivation initially, which might affect

measurement of this outcome. The clients also have complex motivations that are usually

high at the time of contacting the service, such as gaining access to children, which can be

significantly affected by external factors.

Reduced incidents of DVA and repeat incidents: In order to establish changes in rates of DVA,

clients would need to report behaviour following the course. The large number of clients that

completed the behaviour questionnaire, but were then lost to follow up, demonstrate the

difficulties of longitudinal data collection in this field.

Reducing time that victims spend in abusive relationship: This is a particularly difficult

concept to define. Measurement depends on a number of factors including whether the

Economic Evaluation

30

relationship has ended, or abusive behaviour has ceased. In addition, the impact of the course

on these factors is very difficult to attribute.

Lower levels of worry/ Improved family relationships: A number of instances of improved

family relationships were reported by clients. However, this concept could be viewed as a

proximal outcome, or a necessary precursor of outcomes that are more straightforward to

measure and valued (such as reduced need for friends and family to provide accommodation.

Children: improved wellbeing/mental health and fewer children involved in incidents of

DVA: evidence was not collected on incidents of DVA perpetrated on children. Clients

reported positive effects of improved understanding about the impact on children of

witnessing DVA, but as with reduced incidents of DVA, this is very difficult to measure.

However, the perceived impact of DVA on children was collected using the behaviour

questionnaire, which supported the hypothesis that the course had an effect upon the impact

of DVA on children. The large numbers lost to follow up, is a concern in collecting these data.

Improved school attendance: There is a self-reported item on the behaviour questionnaire

to measure school attendance. However, low numbers of completions (especially for follow

up) means that no meaningful data could be collected.

Increased number of referrals to victim services: Mechanisms for linking F4C and victim

services were found to be lacking. Attempts were made to improve these linkages throughout

the duration of the study.

Changes in type of support requested from other agencies (including less crisis support): It

is unlikely that the programme will have a significant effect on the types of work carried out

by other agencies as these services have thresholds and criteria for triggering an intervention,

which are not necessarily dependent on characteristics of the F4C clients.

Preventing escalation to a multi-agency risk assessment conference (MARAC): this outcome

is very challenging to measure in terms of the counterfactual evidence, as these clients

represent a subset of potential MARAC cases, which cannot be identified without them taking

Economic Evaluation

31

part in the course. It is therefore particularly difficult to estimate the effect of the programme

on this outcome.

Reduced number of children requiring adoption: It is unclear as to whether the programme

has an effect on this outcome. Case notes revealed that once actions have begun, attendance

on the course is unlikely to change the course of events leading to children being taken into

care or adopted. Further evidence could be sought through follow-up with Children’s Services

to seek expert opinions and examples of cases where attendance on the programme might

have had an effect. In cases where the authorities are not proceeding to remove children, it

is very difficult to establish the counterfactual (i.e what would have happened without

attendance on the programme).

Reduction in incidents of abusive behaviour reported to police or other agency (Reduced

numbers of police call outs/policing cost reduced rate of domestic abuse): Detailed evidence

from police regarding individual clients was not available during the study. This could be

difficult to acquire for large numbers of service users.

Attendance of employee/perpetrator/ victims at work; performance of perpetrator at

work: This outcome was not systematically collected. Gaining information about victims is

particularly challenging for a voluntary perpetrator programme. Information about clients

would also benefit from longer-term follow up, which is a particular challenge for this client

group.

3.2 SROI Calculations

Tables 3.3 & 3.2 show the SROI ratio of the programme from the start of the programme to

the end of a five-year period of impact calculation. The reported SROI model assumes the only

investments into the programme are those actually input during the study period of two and

a half years. By quarter three of year three, the total value of the investment is therefore fixed

at £382,500, After the first two and a half years, there is zero cost of the programme and the

quarterly benefits begin to decline due to drop off and discounting.

Economic Evaluation

32

The programme does not quite achieve pay back during the study period of two and a half

years. After achieving the benefits of the final quarter, the SROI ratio is £0.96-£1.00, or 96

pence of social value for every pound spent. However, many of the social benefits of the

programme (particularly regarding benefits for children and health and social costs of

substance abuse) are long term and even allowing for drop off and discounting they continue

to accrue. By quarter four in the third year, the SROI ratio becomes positive (£1.10) and the

programme achieves pay back on investment. By the end of year five the SROI ratio is showing

greater than double the return on investment (£2.05).

3.2.1 Sensitivity Analysis

Table 3.2 below shows the counted elements of the SROI model. These values and

assumptions were tested to assess what difference adjustments would make to the final

calculation. Table 3.5 presents low sensitivity analysis. This low estimation is required for the

programme to not demonstrate a positive return over a five-year period. Table 3.6 shows the

returns on investment over two timescales for a programme funded for two and a half years

(i.e. the actual funding period for the evaluation). The Y3Q3 column demonstrates the SROI

ratio immediately after the funding ceased for a two-and-a-half-year long programme. The

Y5Q4 column demonstrates the cumulative effects of benefits that continue to accrue for a

further two and a half years following the two and a half year funding window (total return

over five years).

A particularly sensitive value included in the model, is provision of a route for engagement

with a difficult to reach cohort in order to assess and intervene to address issues such as;

mental and physical health, housing, finance, employment, substance abuse and criminal

behaviour. The English average (2016) inflation adjusted cost per service user contact for drug

outreach services is £104.39 (Table 3.1). This has been included as the value for a single

contact. However, the minimum number of contacts for completers is between 10 and 12,

and even non-completers would be expected to have more than a single contact with a

support worker. The addition of instances of this value makes very large adjustments to the

assessment of benefits. For instance, if two contacts rather than one are applied, the

Economic Evaluation

33

Table 3.2: Counted SROI Values

Outcome Relationship Assumption Value/Qtr Quantity/Qtr Value

Reduced substance

abuse

Prevention of harmful drinker costs

to society

Case notes report recognising problem and

subsequent stopping or moderating

drinking, 3.65 per quarter.

£738.27 3.65 £2,694.69

Reduced substance

abuse

Prevention of harmful drinking:

quality adjusted life years

(13% of 281)= 36.5 cases total over

programme = 3.65 per quarter

£578.81 3.65 £2,112.66

Improved parenting Fiscal savings to public sector

agencies derived from the delivery

of parenting programmes

All completed clients with children n= 66

(6.6 per quarter).

£82.39 6.60 £543.77

Reduced accident and

injury

Reduced A&E attendance of client 2 cases, or 0.2 per quarter £132.00 0.20 £26.40

Reduced need to provide

refuge for victims

Family and friends who help with

accommodation

Assumption that prevented escalation of

DVA (reported by 5 clients) to the point

that the partner feels the need to move

out

£893.17 0.50 £446.59

Reduced demand on

other agencies:

Especially Children’s

Services

F4C staff members assist social

workers

4x1 hour one-to-one plus estimate of

information gathering from the group

sessions at 1 hour per client

£270.00 10.30 £2,781.00

Total £8,605.10

Economic Evaluation

34

Table 3.3: 0 to 2.5 year SROI for a 2.5 year funded programme (281 clients over 10 quarters)

Y1_Q1 Y1_Q2 Y1_Q3 Y1_Q4 Y2_Q1 Y2_Q2 Y2_Q3 Y2_Q4 Y3_Q1 Y3_Q2

Discounted

benefits per

quarter

£0.00 £8,457.10 £16,083.97 £23,055.98 £29,484.54 £35,446.72 £41,000.23 £46,191.09 £51,057.67 £55,632.90

Present Value

(cumulative value

of all benefits)

£0.00 £8,457.10 £24,541.07 £47,597.06 £77,081.60 £112,528.32 £153,528.55 £199,719.64 £250,777.31 £306,410.21

Total value of

Investment/Inputs

£38,250.00 £76,500.00 £114,750.00 £153,000.00 £191,250.00 £229,500.00 £267,750.00 £306,000.00 £344,250.00 £382,500.00

Net Present Value

(value of all

benefits minus

programme costs)

-£38,250.00 -£68,042.90 -£90,208.93 -£105,402.94 -£114,168.40 -£116,971.68 -£114,221.45 -£106,280.36 -£93,472.69 -£76,089.79

SROI £0.00 £0.11 £0.21 £0.31 £0.40 £0.49 £0.57 £0.65 £0.73 £0.80

Economic Evaluation

35

Table 3. 4: 2.5 to 5 year SROI for a 2.5 year funded programme (281 clients over 10 quarters)

Y3_Q3 Y3_Q4 Y4_Q1 Y4_Q2 Y4_Q3 Y4_Q4 Y5_Q1 Y5_Q2 Y5_Q3 Y5_Q4

Discounted

benefits per

quarter

£59,945.53 £56,233.50 £53,061.22 £50,288.80 £47,832.77 £45,639.13 £43,669.87 £41,896.14 £40,294.70 £38,846.05

Present Value

(cumulative value

of all benefits)

£366,355.7

5

£422,589.2

5

£475,650.4

6

£525,939.2

7

£573,772.0

4

£619,411.1

7

£663,081.0

4

£704,977.1

9

£745,271.8

9

£784,117.9

3

Total value of

Investment/Input

s

£382,500.0

0

£382,500.0

0

£382,500.0

0

£382,500.0

0

£382,500.0

0

£382,500.0

0

£382,500.0

0

£382,500.0

0

£382,500.0

0

£382,500.0

0

Net Present Value

(value of all

benefits minus

programme costs)

-£16,144.25 £40,089.25 £93,150.46 £143,439.2

7

£191,272.0

4

£236,911.1

7

£280,581.0

4

£322,477.1

9

£362,771.8

9

£401,617.9

3

SROI £0.96 £1.10 £1.24 £1.38 £1.50 £1.62 £1.73 £1.84 £1.95 £2.05

Economic Evaluation

36

Table 3.5: Low sensitivity analysis

Outcome Relationship Assumption

Reduced substance

abuse

harmful drinker costs to society Increased drop off from 10% to 20% to assume higher relapse rate and reduced

attribution from 100% to 50% to allow for other service interventions having an impact

Reduced substance

abuse

Prevention of harmful drinking:

quality of life adjusted years

Increase drop off from 10% to 20% to assume higher relapse rate and reduced attribution

from 100% to 50% to allow for other service interventions having an impact

Improved parenting Fiscal savings to public sector

agencies derived from the

delivery of parenting programmes

No change to assumptions; there is good evidence that clients learn about and are

strongly affected by the potential psychological harm to children caused by their abusive

behaviour, and respond well to learning about improved parenting. The research evidence

for valuing benefits is good and is conservative in claims about benefits.

Reduced accident and

injury

Reduced A&E attendance of

client

Assume no cases of reduced attendance

Reduced need to

provide refuge for

victims

Family and friends who help with

accommodation

Assume that the course only contributed 50% rather than 100% attribution to this

outcome

Reduced demand on

other agencies:

Especially Children’s

Services

F4C staff members assist social

workers

Assume deadweight increase from 0% to 50% as information could come from alternative

sources

Economic Evaluation

37

Table 3.6: Sensitivity analysis comparisons

Y3-

Q3

Y5-

Q4

Breakeven

point

Original SROI analysis 0.96 2.05 Y3_Q4

Assume no value for housing or A&E 0.93 2.02 Y3-Q4

Increase to 20% drop off for alcohol interventions 0.85 1.72 Y4-Q1

Add deadweight for reduced demand on other services to 50% 0.75 1.55 Y4-Q2

Increase to 20% drop off + reduce attribution to 50% for alcohol

interventions

0.66 1.44 Y4_Q3

Assume no value of reduced demand on other services 0.58 1.10 Y5-Q2

All low SROI assumptions (as shown in table 3.6) 0.46 0.95 -

programme demonstrates a marginal cost saving of £2,743 per year. However, to prevent

double-counting of activities, we have applied the value to the effort required for the first

contact only.

The sensitivity analysis demonstrates that the model returns a positive SROI ratio over five

years except when all assumptions are considerably reduced (table 3.5). Key assumptions that

have important sensitivity implications and therefore would benefit from further

investigation are the effect of the alcohol abuse intervention within the programme, and the

relapse rates of participants. The other assumption that has important implications for the

model, and would benefit from further investigation, is the value of information gathering,

communication and collaboration with other services, specifically children’s services. The

evidence available suggests that the F4C programme delivers a social return on investment of

between £2.05 and £0.95 over five years, for a two and a half year funding period. However,

this estimate could be much higher as there are additional potential benefits, included in the

model, that were not valued as there was no evidence of outcomes available. If ongoing

funding is included in the model, after five years the net present value is forecast at £302,208,

with a SROI ratio of £1.40 and a break-even point of the 1st quarter in year 4.

Economic Evaluation

38

DISCUSSION The SROIA attempts to place values on the benefits of outcomes from the programme; from

the perspective of various stakeholders and society in general. This is a useful approach for

considering the value of interventions from a broader perspective than simply organisational

or individual fiscal effects. The approach is particularly suited to social programmes that have

a wide range of long-term effects. It can be used to make judgements about whether it is

likely that a programme demonstrates a net return to society compared to the societal costs

of delivery.

Voluntary DVA perpetrator programmes are notoriously problematic to evaluate, particularly

regarding obtaining accurate and long term outcome measures, whilst maintaining

confidentiality, gaining consent and working with highly sensitive data from a range of

organisations. For these and other reasons, described in detail above, difficulties were

experienced in measuring and valuing assumptions about expected outcomes. Therefore,

there are many potential outcomes that might have provided additional value to the SROI

analysis, but which were not included in the final calculations. The model should therefore be

considered a conservative estimation of returns on investment; although it should be

recognised that some possible negative influences were also not able to be collected. In

particular, the study did not systematically collect evidence of clients accessing other services

so no attribution to external influences was calculated for the SROI ratio. However, attribution

was deducted in the sensitivity analysis to estimate the effect on the final calculation of

possible external influences on outcomes.

In addition, no displacement was calculated. This is potentially a significant limitation as there

is also the possibility that the programme could result in negative impacts, such as those

resulting from disguised compliance, clients learning new forms of abusive behaviour, and

false feelings of security for victims. In providing details of all outcomes, whether calculated

or not (in Table 3.3), it is possible for users of these evaluation findings to use these key

outcomes in future SROIA, and make adjustments to the calculations to potentially arrive at

more accurate findings for other services. It is also worth noting that little accurate evidence

was available for the rate of drop-off for some of the effects of the programme.

Economic Evaluation

39

Whilst the calculations shown above are for returns calculated for the actual two-and-a-half-

year period of the study, it is also worthwhile to demonstrate what the return might be if the

programme were funded for a longer period of time. The findings of the SROI analysis indicate

that if this programme were funded for five years, the SROI ratio would be a £1.40 return for

each £1.00 invested in the programme, and it would break even in the first quarter of year

four. However, a large number of outcomes that were identified by stakeholders as being

important to take into consideration were not able to be counted.

There are only five outcomes that demonstrate large benefits (harmful drinking costs to

society and the individual, improved parenting, reduced need to provide refuge for victims

and reduced demand on other agencies). There are fourteen outcomes that are not counted,

and therefore the results are highly uncertain. Despite this, a sensitivity analysis showed that

if the programme were only funded for two and a half years and very low estimates of benefits

were applied, then the return after five years would be approaching break-even point at

£0.95. In terms of benefits to society, organisations and individuals, it can therefore be stated

with some certainty that, over the medium to long-term, the programme returns higher value

than the resources required to provide it.

The voluntary perpetrator programme has a number of outcome values that are realised over

the long term. For instance, improved parenting can result in social value even once children

have become adults. There is strong evidence that the legacy of poor parenting can contribute

to significant social costs later in life, such as the costs related to victims of crime.

Interventions when children are five years old are often calculated on a twenty-five-year

cycle, until the child is thirty years old. However, for the purpose of this study only a five-year

period is considered, which means that some benefits may be underestimated.

One of the key benefits of SROI methodology is that it does not apply normative principles,

but rather is contextually determined. Users of these findings would be encouraged to use

their judgement and established SROI methods to understand the validity of the benefits

counted in this model for their own situation.

Economic Evaluation

40

RECOMMENDATIONS FOR FUTURE RESEARCH

Future economic evaluations of voluntary DVA perpetrator programmes should consider the

difficulties encountered with this study, and seek to identify measures and data collection

mechanisms for a wide range of outcomes. Many have successfully been developed for this

study. However, there are considerable advances that could still be made; some of which are

described in detail in this report and in the companion report for the full evaluation.

The current difficulties of obtaining good quality outcomes data for these types of services

puts in question the desire from policy-makers and commissioners to have evidence of

returns on investment (ROI). It should be recognised that there are significant difficulties

involved in demonstrating ROI and improvements need to be made regarding putting

measures in place. It should also be recognised that interpretation of findings requires a deep

understanding of the methodology, including the strengths and weaknesses.

Two pervasive problems for evaluations of this type of programme are the loss of clients to

follow-up and difficulties in obtaining outcome measures for victims and families. To an

extent, these could be gathered using proxy measures during the window of engagement with

the service-user. For instance, asking questions about disruption to children’s schooling and

the input of other services. For the purposes of affordable evaluation and monitoring of

services, this could be a more cost-effective approach to understanding benefits, although

the disadvantages of relying on clients’ self-reports are clear. Another important area of

further research is the resilience of benefits, which would allow greater confidence in the

values used to calculate drop-off. This evidence could also help in the design of interventions

to more appropriately support perpetrators following the initial intervention.

Future studies should also be aware of the potential for negative impacts, such as those

resulting from disguised compliance, clients learning new forms of abusive behaviour, and

false feelings of security for victims. However, it is not currently apparent how these negative

impacts could be measured. The analysis that has been carried out allows clear

recommendations, including (a) useful approaches for those for collecting and measuring

outcomes and (b) areas that would benefit from future research.

Economic Evaluation

41

REFERENCES Akoensi, T. D., Koehler, J. A., Lösel, F., & Humphreys, D. K. (2013). Domestic Violence

Perpetrator Programs in Europe, Part II A Systematic Review of the State of Evidence. International journal of offender therapy and comparative criminology, 57(10), 1206-1225.

Alhabib, S., Nur, U., & Jones, R. (2010). Domestic violence against women: Systematic review of prevalence studies. Journal of family violence, 25(4), 369-382.

Ali, P., Ariss, S., Powell, K., Crowder, M., & Cooper, R. (2017). Doncaster Domestic Abuse Perpetrator Programme: Evaluation Report, 2017. Retrieved from https://www.sheffield.ac.uk/snm/research/doncaster

Arvidson, M., Battye, F., & Salisbury, D. (2014). The social return on investment in community befriending. Int J Public Sector Manag., 27. doi:10.1108/ijpsm-03-2013-0045

Banke-Thomas, A. O., Madaj, B., Charles, A., & van den Broek, N. (2015). Social Return on Investment (SROI) methodology to account for value for money of public health interventions: a systematic review. BMC Public Health, 15(1), 582. doi:10.1186/s12889-015-1935-7

Bellis, M. A., Hughes, K., Perkins, C., & Bennett, A. (2012). Protecting people promoting health: a public health approach to violence prevention for England. Liverpool: Department of Health.

Bonomi, A. E., Anderson, M. L., Rivara, F. P., & Thompson, R. S. (2009). Health Care Utilization and Costs Associated with Physical and Nonphysical‐Only Intimate Partner Violence. Health services research, 44(3), 1052-1067.

Cabinet Office. (2016). Social Value Act: information and resources. London: Cabinet Office, UK Retrieved from https://www.gov.uk/government/publications/social-value-act-information-and-resources.

DiClemente, C. C., & Prochaska, J. O. (1982). Self-change and therapy change of smoking behavior: A comparison of processes of change in cessation and maintenance. Addictive behaviors, 7(2), 133-142.

Doncaster Domestic Abuse Service. (2016). Doncaster Domestic Abuse Strategy 2016-2020. Retrieved from http://doncasterdomesticabuse.co.uk/information-professionals

Farmer, A., & Tiefenthaler, J. (1997). An Economic Analysis of Domestic Violence. Review of Social Economy, 55(3), 337-358. doi:10.1080/00346769700000004

Finney, J. W., & Moos, R. H. (1991). The long-term course of treated alcoholism: I. Mortality, relapse and remission rates and comparisons with community controls. Journal of studies on alcohol, 52(1), 44-54.

Hamilton, L., Koehler, J. A., & Lösel, F. A. (2012). Domestic violence perpetrator programs in Europe, part I: A survey of current practice. International journal of offender therapy and comparative criminology, 0306624X12469506.

Miller, W. R., & Rollnick, S. (2012). Motivational interviewing: Helping people change: Guilford press.

Nef consultancy. (2013). Social valuation of Refuge services for survivors of domestic violence. nef consultancy report for Refuge Retrieved from London: http://www.refuge.org.uk/files/Refuge-SROI-report-25-09-13-NCV2.pdf

Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2009). A guide to social return on investment. London: Office of the Third Sector, The Cabinet Office.

Economic Evaluation

42

Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2012). A Guide to Social Return on Investment. London: The Cabinet Office.

Perfect Moment. (2010). Strength to Change Return on Investment Study. Retrieved from Cornwall: http://www.ccrm.org.uk/images/docs/8.3apm%20stc%20roi%20exec%20summary%2030nov10.pdf

Prochaska, J. O. (2013). Transtheoretical model of behavior change Encyclopedia of behavioral medicine (pp. 1997-2000): Springer.

Prochaska, J. O., & DiClemente, C. C. (1986). Toward a comprehensive model of change Treating addictive behaviors (pp. 3-27): Springer.

Rollnick, S., & Miller, W. R. (1995). What is motivational interviewing? Behavioural and cognitive Psychotherapy, 23(04), 325-334.

Rotheroe, N., & Richards, A. (2007). Social return on investment and social enterprise: transparent accountability for sustainable development. Social Enterprise Journal., 3. doi:10.1108/17508610780000720

UK Home Office. (2013). Domestic violence and abuse. Retrieved from https://www.gov.uk/guidance/domestic-violence-and-abuse#history

Willis, E., Semple, A. C., & de Waal, H. (2016). Quantifying the benefits of peer support for people with dementia: A Social Return on Investment (SROI) study. Dementia, 1471301216640184.

Wood, C., & Leighton, D. (2010). Measuring social value: the gap between policy and practice. Undercurrent(2), 7.

World Health Organization. (2014). Violence against women. Retrieved from http://www.who.int/mediacentre/factsheets/fs239/en/

Wright, S., Nelson, J. D., Cooper, J. M., & Murphy, S. (2009). An evaluation of the transport to employment (T2E) scheme in Highland Scotland using social return on investment (SROI). Journal of Transport Geography, 17(6), 457-467.

Yodanis, C. L., Godenzi, A., & Stanko, E. A. (2000). The benefits of studying costs: A review and agenda for studies on the economic costs of violence against women. Policy Studies, 21(3), 263-276.

The research was funded by the NIHR School for Public Health Research

(SPHR) Public Health Practice Evaluation Scheme (PHPES) The views

expressed are those of the author(s) and not necessarily those of the

NHS, the NIHR or the Department of Health.

The NIHR School for Public Health Research is a partnership between the Universities of Sheffield,

Bristol, Cambridge, Imperial and University College London; The London School for Hygiene and

Tropical Medicine (LSHTM); the LiLaC collaboration between the Universities of Liverpool and

Lancaster and Fuse; The Centre for Translational Research in Public Health, a collaboration between

Newcastle, Durham, Northumbria, Sunderland and Teesside Universities