55
Public Value How can it be measured, managed and grown? Geoff Mulgan, Jonathan Breckon, Mariola Tarrega, Hasan Bakhshi, John Davies, Halima Khan, Annie Finnis May 2019

Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grownGeoff Mulgan Jonathan Breckon Mariola Tarrega Hasan Bakhshi John Davies Halima Khan Annie Finnis

May 2019

About Nesta

Nesta is an innovation foundation For us innovation means turning bold ideas into reality and changing lives for the better

We use our expertise skills and funding in areas where there are big challenges facing society

Nesta is based in the UK and supported by a financial endowment We work with partners around the globe to bring bold ideas to life to change the world for good

wwwnestaorguk

If yoursquod like this publication in an alternative format such as Braille large print or audio please contact us at informationnestaorguk

Design Green Doe Graphic Design Ltd

Public ValueHow can it be measured managed and grownSummary 4

Background Searching for value 6

A quick survey of the landscape of methods 8 Services outcomes and justice 10

Value in the real world Case studies from the frontlines of public value 111 The What Works Movement Bringing more rigorous views of impact 11 and value into public services

No common currency of value 14 Valuing children in care 14 How to collect trustworthy data 15 Citizens measuring creating and evaluating evidence 17 Differing time horizons 17 Conclusion 19

2 Measuring value in culture and heritage 20

Using wellbeing analysis 24 Policy recommendations Using existing valuation frameworks more 24 effectively and involving the public in valuation in new ways

3 Realising the value in health 27

When it comes to health people value more than just clinical outcomes 27 People make decisions every day that affect their health and wellbeing 28 Value for people 30 Value for the system 31 Wider social value 34 A broader view of value 34 Conclusion 36

Some general lessons 37

Social and public value Effective demand and effective supply 38Horizons in time 43Linking public value to public finance 44Lessons for social and public value 47

Annex Cost and value in the UK What Works Centres 49

Endnotes 53

4

Public Value How can it be measured managed and grown

SummaryRecent decades have brought periodic waves of interest in finding better ways of mapping and measuring public and social value - outcomes institutions and services that are valued by the public but not easy to count in the way that the monetary value of cars or computers can be Governments and parliaments have sometimes tried to encourage this In the UK for example the Social Value Act explicitly encouraged broader notions of value as have recent revisions of the Treasury Green Book used to assess public spending choices

But these generally remain underdeveloped and lack influence compared to more traditional market valuation methods Within government there are still far more developed means for measuring investment in physical things - railways roads airports - than in intangibles like reducing social isolation improving health or the arts This almost certainly continues to skew public spending

Itrsquos obviously wrong to claim that you canrsquot manage what you canrsquot measure But the lack of well-established metrics for public value in all its forms makes it harder to compare alternatives and continues to bias public spending against prevention rather than cure against indirect benefits relative to direct ones and against intangibles relative to physical objects

This paper sets out alternatives It brings together views from Nesta on better ways of mapping and measuring public value It builds on work Nesta has done in many fields - from health and culture to public services - to find more rounded and realistic ways of capturing the many dimensions of value created by public action It is relevant to our work influencing governments and charities as well as to our own work as a funder since our status as a charity commits us to creating public benefit

Our aim in this work is to make value more transparent and more open to interrogation whether that concerns libraries bicycle lanes museums primary health services or training programmes for the unemployed We recognise that value may come from government action it can also be created by others in civil society and business And we recognise that value can often be complex whether in terms of who benefits or how it relates to values as well as more technical issues such as what discount rates to apply

But unless value is attended to explicitly we risk ending up with unhappy results This is why we have focused so much on how to both understand as well as create value through setting up What Works Centres and the Alliance for Useful Evidence through impact investment funds that use more precise metrics for social impact through projects in health that recognise what is important to people themselves through projects to measure the value of culture and through programmes like lsquoInnovate to Saversquo in Wales that aims to achieve cashable and other savings from innovations

5

Public Value How can it be measured managed and grown

This paper documents work in progress but we hope its concepts and applications will be useful It shows that

1 Despite the tensions and dilemmas public value - in all its forms - can be mapped and often measured in ways that are useful for shaping decisions

2 The best valuation methods avoid the twin errors of on the one hand believing that a single measure can adequately capture complex values and on the other believing that decisions should just be guided by hunch or intuition

3 Public value like value in markets - is not a static fact but arises from the interaction of changing options for supply and changing demands and public priorities

4 The greatest use of measures of public value comes not just from more detailed analysis but from the conversations and negotiations they prompt The key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

5 Public value methods should help governments decide on actions now that can deliver benefits in the long term whether on climate change early childhood or healthcare This requires new ways of managing public investment but also a much more sophisticated approach to discount rates

6 The debate about public value needs to be tied into much more energetic reform of public finance methods which need to become more transparent more focused on outcomes better suited to experiment and innovation and better shaped for public engagement If this doesnrsquot happen anachronistic methods for managing finance will crush imaginative attempts to map and measure public value

7 Valuation has limits in a democracy it should be possible for citizens or politicians to decide that they donrsquot want the state involved either in measuring or seeking to influence important areas of life including families and communities

This field is more of a craft than a science But itrsquos one with a growing range of tools that can in time evolve to become more reliable and rigorous We advocate more work to seek out value and where possible to measure it and use it to guide spending decisions and we believe that this will increasingly involve new data sources including public administrative data We hope that governments will commit themselves to using public value to guide their decisions and future budgets and to catalyse long-overdue reform to the processes of public finance

Public Value How can it be measured managed and grown

6

Background Searching for value By Geoff Mulgan

Until the 1990s international accounting conventions assumed that public sector productivity never improved the value of public provision was assumed to be equal to its cost Yet in a period of greater public pressure for results and value for money that position became untenable

For a time the main focus was on critique ndash showing the limitations of mainstream economics and methods like cost-benefit analysis and how inadequately they captured externalities and social benefits

But over the last 20 years serious attention has turned to finding better alternatives (though critiques can still pull in a big audience) Public bodies like the BBC commissioned detailed studies of the public value they were creating back in the 2000s and for a time the Cabinet Office promoted new methods of assessing public value These methods aim to measure not just what public interventions cost but also provide more rounded accounts of what value they create and just as important what value civil society can provide for public commissioners and purchasers There have been advances both in economic approaches ndash like Cost Benefit Analysis (CBA) and the UK Treasury Green Book ndash and non-economic ones that are mirrored in many countries around the world Ambitious ideas are also being developed for more precise measures of social and other goods1

Therersquos been a parallel push in civil society mainly driven by funders wanting better measures of social impact and social returns better measures for the impact in lsquoimpact investmentrsquo and better ways to compare the results of spending in different fields A plethora of organisations have been set up to help map and measure value and impact and ndash like Global Impact Investment Network ndash to establish some common standards for fields like clean energy or healthcare Others offer to measure social value

At least four distinct communities have been having very similar discussions about value though with surprisingly little cross-pollination statisticians attempting to measure wellbeing and outcomes of all kinds (for example brought together in the OECD Better Life Index) economists applying measurement tools to new fields such as carbon and natural capital philanthropists and NGOs interested in impact investment and activists using new measures as a tool for social change (for example measuring or revealing carbon footprints or social neglect)

Not too far from these there has also been an explosion of methods for measuring intangibles ndash the value of ideas knowledge creativity ndash many of which Nesta has been closely involved in Some measure innovation spending and the value it creates others the inputs and outputs of the creative economy Stian Westlakersquos book with Jonathan Haskel Capitalism without Capital provides an excellent overview

Public Value How can it be measured managed and grown

7

Better metrics do not of themselves deliver better outcomes You canrsquot fatten a pig by weighing it But if you donrsquot have some means of weighing it you may find yourself unable to persuade others that itrsquos as fat as you believe

Many methods try to put a price on value The idea that the value to be gained from a new training programme can be directly compared with the value from a health screening programme or water conservation is immediately appealing to busy bureaucrats and ministers who have to justify spending money on one area over another as it is to foundations

So how can this be done Traditional economic valuation methods (based on utilitarian principles) which try to monetise public value usually draw either on what people say they would pay for a service or outcome (lsquostated preferencersquo) or on the choices people have made in related fields (lsquorevealed preferencersquo) These methods also try to adjust the estimated benefits of public services for quality of outcomes ndash for example comparing school exam results or the success of operations As they deliver commensurable estimates economic methods imply that the benefits of disparate outcomes can be compared and added up Other economic methods such as subjective wellbeing2 compare public policy actions by estimating the extra income people would need to achieve an equivalent gain in life satisfaction

Decision-making becomes a lot easier if any option can be turned into a comparable number and there have been many attempts to create such tools Governments would love to be able to use a single metric to compare spending on a scheme for housing the homeless primary education bridges and defence though none have come even remotely close to having such a tool Other attempts include CBA and in the social sector social return on investment (SROI) and more recently Y Analytics and the Impact Multiple of Money a single number now being promoted to adjust company valuations to take account of all their impacts

These are all well-intentioned But paying too much attention to monetary equivalence can lead to bad decisions and Irsquove become sceptical of moves to pretend that a single number can compare very different activities and outcomes These fall apart intellectually and practically for many reasons (I set out the arguments in more detail in this piece)

Some are reasons of principle (the plurality of approaches to valuation in different fields or spheres) some reasons fall on the boundaries of theory and practice (such as what discount rates to apply to different activities) and some are reasons of practicality (the experience that different methods used to assess value can generate wildly different numbers and often miss out what people turn out to value most (because people may not be willing to countenance trading off the things they care about most)3 As one recent book put it value lsquois neither an intrinsic quality of an object nor a reflection of a subjects preferences but rather something that is organized and brought into existence through mechanisms technologies and practices of valuationrsquo4 So our attention needs to focus on what these mechanisms are and whether they are fit for purpose and how they can be made useful rather than misleading

Public Value How can it be measured managed and grown

8

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society providing a bridge between the complex patterns of public demands and needs as expressed through political and other processes and the changing production systems that keep people healthy educated or safe

Cost-benefit analysis (CBA) has been the most widely used mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs)5 CBA has been subject to criticism for decades but continues to be used and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA

In philanthropy and civil society many methods are used some linked into standards of evidence of the kind Nesta has promoted These aim to capture data on impacts where possible to draw on longitudinal data sets and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1000 different methods in use) Often these attempt to provide versions of lsquobalanced scorecardsrsquo or outcome stars to map value in multiple dimensions

There are also the many applications of public sector welfare economics such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions Detailed methods have been in use for several decades to estimate the direct costs of an action (eg a drug treatment programme) the probability of it working and the likely impact on future crime rates hospital admissions or welfare payments

Analysis of this kind can be very powerful In the USA for example researchers identified what they called lsquomillion dollar blocksrsquo where the costs associated with criminals topped the million-dollar mark In principle good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime

These analyses can also be made highly sophisticated They can for example be adapted to reflect income distribution on the principle that an extra dollar or an extra unit of utility is worth more to a poor person than a rich one (since 2003 the UK Governmentrsquos Green Book has required all appraisals to include distributional effects) or time effects (on the principle that if growth continues the marginal value of an extra pound will be worth less in 50 yearsrsquo time simply because average incomes will be higher)

But all measurements of complex effects are inherently hard and they can be costly The benefits tend to become more dispersed over time affecting ever more agencies in ever more uncertain ways Social science is rarely robust enough to make hard predictions about what causes will lead to what effects ndash there are usually far too many variables involved Meanwhile standard cost benefit models apply discount rates to these gains usually based on prevailing commercial interest rates which renders a benefit in a generationrsquos time virtually worthless

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable But in many of the most important fields for government action or philanthropy ndash like childcare crime prevention or schooling ndash the public are divided over values as well as value For most people for example there is an

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 2: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

About Nesta

Nesta is an innovation foundation For us innovation means turning bold ideas into reality and changing lives for the better

We use our expertise skills and funding in areas where there are big challenges facing society

Nesta is based in the UK and supported by a financial endowment We work with partners around the globe to bring bold ideas to life to change the world for good

wwwnestaorguk

If yoursquod like this publication in an alternative format such as Braille large print or audio please contact us at informationnestaorguk

Design Green Doe Graphic Design Ltd

Public ValueHow can it be measured managed and grownSummary 4

Background Searching for value 6

A quick survey of the landscape of methods 8 Services outcomes and justice 10

Value in the real world Case studies from the frontlines of public value 111 The What Works Movement Bringing more rigorous views of impact 11 and value into public services

No common currency of value 14 Valuing children in care 14 How to collect trustworthy data 15 Citizens measuring creating and evaluating evidence 17 Differing time horizons 17 Conclusion 19

2 Measuring value in culture and heritage 20

Using wellbeing analysis 24 Policy recommendations Using existing valuation frameworks more 24 effectively and involving the public in valuation in new ways

3 Realising the value in health 27

When it comes to health people value more than just clinical outcomes 27 People make decisions every day that affect their health and wellbeing 28 Value for people 30 Value for the system 31 Wider social value 34 A broader view of value 34 Conclusion 36

Some general lessons 37

Social and public value Effective demand and effective supply 38Horizons in time 43Linking public value to public finance 44Lessons for social and public value 47

Annex Cost and value in the UK What Works Centres 49

Endnotes 53

4

Public Value How can it be measured managed and grown

SummaryRecent decades have brought periodic waves of interest in finding better ways of mapping and measuring public and social value - outcomes institutions and services that are valued by the public but not easy to count in the way that the monetary value of cars or computers can be Governments and parliaments have sometimes tried to encourage this In the UK for example the Social Value Act explicitly encouraged broader notions of value as have recent revisions of the Treasury Green Book used to assess public spending choices

But these generally remain underdeveloped and lack influence compared to more traditional market valuation methods Within government there are still far more developed means for measuring investment in physical things - railways roads airports - than in intangibles like reducing social isolation improving health or the arts This almost certainly continues to skew public spending

Itrsquos obviously wrong to claim that you canrsquot manage what you canrsquot measure But the lack of well-established metrics for public value in all its forms makes it harder to compare alternatives and continues to bias public spending against prevention rather than cure against indirect benefits relative to direct ones and against intangibles relative to physical objects

This paper sets out alternatives It brings together views from Nesta on better ways of mapping and measuring public value It builds on work Nesta has done in many fields - from health and culture to public services - to find more rounded and realistic ways of capturing the many dimensions of value created by public action It is relevant to our work influencing governments and charities as well as to our own work as a funder since our status as a charity commits us to creating public benefit

Our aim in this work is to make value more transparent and more open to interrogation whether that concerns libraries bicycle lanes museums primary health services or training programmes for the unemployed We recognise that value may come from government action it can also be created by others in civil society and business And we recognise that value can often be complex whether in terms of who benefits or how it relates to values as well as more technical issues such as what discount rates to apply

But unless value is attended to explicitly we risk ending up with unhappy results This is why we have focused so much on how to both understand as well as create value through setting up What Works Centres and the Alliance for Useful Evidence through impact investment funds that use more precise metrics for social impact through projects in health that recognise what is important to people themselves through projects to measure the value of culture and through programmes like lsquoInnovate to Saversquo in Wales that aims to achieve cashable and other savings from innovations

5

Public Value How can it be measured managed and grown

This paper documents work in progress but we hope its concepts and applications will be useful It shows that

1 Despite the tensions and dilemmas public value - in all its forms - can be mapped and often measured in ways that are useful for shaping decisions

2 The best valuation methods avoid the twin errors of on the one hand believing that a single measure can adequately capture complex values and on the other believing that decisions should just be guided by hunch or intuition

3 Public value like value in markets - is not a static fact but arises from the interaction of changing options for supply and changing demands and public priorities

4 The greatest use of measures of public value comes not just from more detailed analysis but from the conversations and negotiations they prompt The key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

5 Public value methods should help governments decide on actions now that can deliver benefits in the long term whether on climate change early childhood or healthcare This requires new ways of managing public investment but also a much more sophisticated approach to discount rates

6 The debate about public value needs to be tied into much more energetic reform of public finance methods which need to become more transparent more focused on outcomes better suited to experiment and innovation and better shaped for public engagement If this doesnrsquot happen anachronistic methods for managing finance will crush imaginative attempts to map and measure public value

7 Valuation has limits in a democracy it should be possible for citizens or politicians to decide that they donrsquot want the state involved either in measuring or seeking to influence important areas of life including families and communities

This field is more of a craft than a science But itrsquos one with a growing range of tools that can in time evolve to become more reliable and rigorous We advocate more work to seek out value and where possible to measure it and use it to guide spending decisions and we believe that this will increasingly involve new data sources including public administrative data We hope that governments will commit themselves to using public value to guide their decisions and future budgets and to catalyse long-overdue reform to the processes of public finance

Public Value How can it be measured managed and grown

6

Background Searching for value By Geoff Mulgan

Until the 1990s international accounting conventions assumed that public sector productivity never improved the value of public provision was assumed to be equal to its cost Yet in a period of greater public pressure for results and value for money that position became untenable

For a time the main focus was on critique ndash showing the limitations of mainstream economics and methods like cost-benefit analysis and how inadequately they captured externalities and social benefits

But over the last 20 years serious attention has turned to finding better alternatives (though critiques can still pull in a big audience) Public bodies like the BBC commissioned detailed studies of the public value they were creating back in the 2000s and for a time the Cabinet Office promoted new methods of assessing public value These methods aim to measure not just what public interventions cost but also provide more rounded accounts of what value they create and just as important what value civil society can provide for public commissioners and purchasers There have been advances both in economic approaches ndash like Cost Benefit Analysis (CBA) and the UK Treasury Green Book ndash and non-economic ones that are mirrored in many countries around the world Ambitious ideas are also being developed for more precise measures of social and other goods1

Therersquos been a parallel push in civil society mainly driven by funders wanting better measures of social impact and social returns better measures for the impact in lsquoimpact investmentrsquo and better ways to compare the results of spending in different fields A plethora of organisations have been set up to help map and measure value and impact and ndash like Global Impact Investment Network ndash to establish some common standards for fields like clean energy or healthcare Others offer to measure social value

At least four distinct communities have been having very similar discussions about value though with surprisingly little cross-pollination statisticians attempting to measure wellbeing and outcomes of all kinds (for example brought together in the OECD Better Life Index) economists applying measurement tools to new fields such as carbon and natural capital philanthropists and NGOs interested in impact investment and activists using new measures as a tool for social change (for example measuring or revealing carbon footprints or social neglect)

Not too far from these there has also been an explosion of methods for measuring intangibles ndash the value of ideas knowledge creativity ndash many of which Nesta has been closely involved in Some measure innovation spending and the value it creates others the inputs and outputs of the creative economy Stian Westlakersquos book with Jonathan Haskel Capitalism without Capital provides an excellent overview

Public Value How can it be measured managed and grown

7

Better metrics do not of themselves deliver better outcomes You canrsquot fatten a pig by weighing it But if you donrsquot have some means of weighing it you may find yourself unable to persuade others that itrsquos as fat as you believe

Many methods try to put a price on value The idea that the value to be gained from a new training programme can be directly compared with the value from a health screening programme or water conservation is immediately appealing to busy bureaucrats and ministers who have to justify spending money on one area over another as it is to foundations

So how can this be done Traditional economic valuation methods (based on utilitarian principles) which try to monetise public value usually draw either on what people say they would pay for a service or outcome (lsquostated preferencersquo) or on the choices people have made in related fields (lsquorevealed preferencersquo) These methods also try to adjust the estimated benefits of public services for quality of outcomes ndash for example comparing school exam results or the success of operations As they deliver commensurable estimates economic methods imply that the benefits of disparate outcomes can be compared and added up Other economic methods such as subjective wellbeing2 compare public policy actions by estimating the extra income people would need to achieve an equivalent gain in life satisfaction

Decision-making becomes a lot easier if any option can be turned into a comparable number and there have been many attempts to create such tools Governments would love to be able to use a single metric to compare spending on a scheme for housing the homeless primary education bridges and defence though none have come even remotely close to having such a tool Other attempts include CBA and in the social sector social return on investment (SROI) and more recently Y Analytics and the Impact Multiple of Money a single number now being promoted to adjust company valuations to take account of all their impacts

These are all well-intentioned But paying too much attention to monetary equivalence can lead to bad decisions and Irsquove become sceptical of moves to pretend that a single number can compare very different activities and outcomes These fall apart intellectually and practically for many reasons (I set out the arguments in more detail in this piece)

Some are reasons of principle (the plurality of approaches to valuation in different fields or spheres) some reasons fall on the boundaries of theory and practice (such as what discount rates to apply to different activities) and some are reasons of practicality (the experience that different methods used to assess value can generate wildly different numbers and often miss out what people turn out to value most (because people may not be willing to countenance trading off the things they care about most)3 As one recent book put it value lsquois neither an intrinsic quality of an object nor a reflection of a subjects preferences but rather something that is organized and brought into existence through mechanisms technologies and practices of valuationrsquo4 So our attention needs to focus on what these mechanisms are and whether they are fit for purpose and how they can be made useful rather than misleading

Public Value How can it be measured managed and grown

8

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society providing a bridge between the complex patterns of public demands and needs as expressed through political and other processes and the changing production systems that keep people healthy educated or safe

Cost-benefit analysis (CBA) has been the most widely used mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs)5 CBA has been subject to criticism for decades but continues to be used and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA

In philanthropy and civil society many methods are used some linked into standards of evidence of the kind Nesta has promoted These aim to capture data on impacts where possible to draw on longitudinal data sets and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1000 different methods in use) Often these attempt to provide versions of lsquobalanced scorecardsrsquo or outcome stars to map value in multiple dimensions

There are also the many applications of public sector welfare economics such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions Detailed methods have been in use for several decades to estimate the direct costs of an action (eg a drug treatment programme) the probability of it working and the likely impact on future crime rates hospital admissions or welfare payments

Analysis of this kind can be very powerful In the USA for example researchers identified what they called lsquomillion dollar blocksrsquo where the costs associated with criminals topped the million-dollar mark In principle good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime

These analyses can also be made highly sophisticated They can for example be adapted to reflect income distribution on the principle that an extra dollar or an extra unit of utility is worth more to a poor person than a rich one (since 2003 the UK Governmentrsquos Green Book has required all appraisals to include distributional effects) or time effects (on the principle that if growth continues the marginal value of an extra pound will be worth less in 50 yearsrsquo time simply because average incomes will be higher)

But all measurements of complex effects are inherently hard and they can be costly The benefits tend to become more dispersed over time affecting ever more agencies in ever more uncertain ways Social science is rarely robust enough to make hard predictions about what causes will lead to what effects ndash there are usually far too many variables involved Meanwhile standard cost benefit models apply discount rates to these gains usually based on prevailing commercial interest rates which renders a benefit in a generationrsquos time virtually worthless

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable But in many of the most important fields for government action or philanthropy ndash like childcare crime prevention or schooling ndash the public are divided over values as well as value For most people for example there is an

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 3: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public ValueHow can it be measured managed and grownSummary 4

Background Searching for value 6

A quick survey of the landscape of methods 8 Services outcomes and justice 10

Value in the real world Case studies from the frontlines of public value 111 The What Works Movement Bringing more rigorous views of impact 11 and value into public services

No common currency of value 14 Valuing children in care 14 How to collect trustworthy data 15 Citizens measuring creating and evaluating evidence 17 Differing time horizons 17 Conclusion 19

2 Measuring value in culture and heritage 20

Using wellbeing analysis 24 Policy recommendations Using existing valuation frameworks more 24 effectively and involving the public in valuation in new ways

3 Realising the value in health 27

When it comes to health people value more than just clinical outcomes 27 People make decisions every day that affect their health and wellbeing 28 Value for people 30 Value for the system 31 Wider social value 34 A broader view of value 34 Conclusion 36

Some general lessons 37

Social and public value Effective demand and effective supply 38Horizons in time 43Linking public value to public finance 44Lessons for social and public value 47

Annex Cost and value in the UK What Works Centres 49

Endnotes 53

4

Public Value How can it be measured managed and grown

SummaryRecent decades have brought periodic waves of interest in finding better ways of mapping and measuring public and social value - outcomes institutions and services that are valued by the public but not easy to count in the way that the monetary value of cars or computers can be Governments and parliaments have sometimes tried to encourage this In the UK for example the Social Value Act explicitly encouraged broader notions of value as have recent revisions of the Treasury Green Book used to assess public spending choices

But these generally remain underdeveloped and lack influence compared to more traditional market valuation methods Within government there are still far more developed means for measuring investment in physical things - railways roads airports - than in intangibles like reducing social isolation improving health or the arts This almost certainly continues to skew public spending

Itrsquos obviously wrong to claim that you canrsquot manage what you canrsquot measure But the lack of well-established metrics for public value in all its forms makes it harder to compare alternatives and continues to bias public spending against prevention rather than cure against indirect benefits relative to direct ones and against intangibles relative to physical objects

This paper sets out alternatives It brings together views from Nesta on better ways of mapping and measuring public value It builds on work Nesta has done in many fields - from health and culture to public services - to find more rounded and realistic ways of capturing the many dimensions of value created by public action It is relevant to our work influencing governments and charities as well as to our own work as a funder since our status as a charity commits us to creating public benefit

Our aim in this work is to make value more transparent and more open to interrogation whether that concerns libraries bicycle lanes museums primary health services or training programmes for the unemployed We recognise that value may come from government action it can also be created by others in civil society and business And we recognise that value can often be complex whether in terms of who benefits or how it relates to values as well as more technical issues such as what discount rates to apply

But unless value is attended to explicitly we risk ending up with unhappy results This is why we have focused so much on how to both understand as well as create value through setting up What Works Centres and the Alliance for Useful Evidence through impact investment funds that use more precise metrics for social impact through projects in health that recognise what is important to people themselves through projects to measure the value of culture and through programmes like lsquoInnovate to Saversquo in Wales that aims to achieve cashable and other savings from innovations

5

Public Value How can it be measured managed and grown

This paper documents work in progress but we hope its concepts and applications will be useful It shows that

1 Despite the tensions and dilemmas public value - in all its forms - can be mapped and often measured in ways that are useful for shaping decisions

2 The best valuation methods avoid the twin errors of on the one hand believing that a single measure can adequately capture complex values and on the other believing that decisions should just be guided by hunch or intuition

3 Public value like value in markets - is not a static fact but arises from the interaction of changing options for supply and changing demands and public priorities

4 The greatest use of measures of public value comes not just from more detailed analysis but from the conversations and negotiations they prompt The key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

5 Public value methods should help governments decide on actions now that can deliver benefits in the long term whether on climate change early childhood or healthcare This requires new ways of managing public investment but also a much more sophisticated approach to discount rates

6 The debate about public value needs to be tied into much more energetic reform of public finance methods which need to become more transparent more focused on outcomes better suited to experiment and innovation and better shaped for public engagement If this doesnrsquot happen anachronistic methods for managing finance will crush imaginative attempts to map and measure public value

7 Valuation has limits in a democracy it should be possible for citizens or politicians to decide that they donrsquot want the state involved either in measuring or seeking to influence important areas of life including families and communities

This field is more of a craft than a science But itrsquos one with a growing range of tools that can in time evolve to become more reliable and rigorous We advocate more work to seek out value and where possible to measure it and use it to guide spending decisions and we believe that this will increasingly involve new data sources including public administrative data We hope that governments will commit themselves to using public value to guide their decisions and future budgets and to catalyse long-overdue reform to the processes of public finance

Public Value How can it be measured managed and grown

6

Background Searching for value By Geoff Mulgan

Until the 1990s international accounting conventions assumed that public sector productivity never improved the value of public provision was assumed to be equal to its cost Yet in a period of greater public pressure for results and value for money that position became untenable

For a time the main focus was on critique ndash showing the limitations of mainstream economics and methods like cost-benefit analysis and how inadequately they captured externalities and social benefits

But over the last 20 years serious attention has turned to finding better alternatives (though critiques can still pull in a big audience) Public bodies like the BBC commissioned detailed studies of the public value they were creating back in the 2000s and for a time the Cabinet Office promoted new methods of assessing public value These methods aim to measure not just what public interventions cost but also provide more rounded accounts of what value they create and just as important what value civil society can provide for public commissioners and purchasers There have been advances both in economic approaches ndash like Cost Benefit Analysis (CBA) and the UK Treasury Green Book ndash and non-economic ones that are mirrored in many countries around the world Ambitious ideas are also being developed for more precise measures of social and other goods1

Therersquos been a parallel push in civil society mainly driven by funders wanting better measures of social impact and social returns better measures for the impact in lsquoimpact investmentrsquo and better ways to compare the results of spending in different fields A plethora of organisations have been set up to help map and measure value and impact and ndash like Global Impact Investment Network ndash to establish some common standards for fields like clean energy or healthcare Others offer to measure social value

At least four distinct communities have been having very similar discussions about value though with surprisingly little cross-pollination statisticians attempting to measure wellbeing and outcomes of all kinds (for example brought together in the OECD Better Life Index) economists applying measurement tools to new fields such as carbon and natural capital philanthropists and NGOs interested in impact investment and activists using new measures as a tool for social change (for example measuring or revealing carbon footprints or social neglect)

Not too far from these there has also been an explosion of methods for measuring intangibles ndash the value of ideas knowledge creativity ndash many of which Nesta has been closely involved in Some measure innovation spending and the value it creates others the inputs and outputs of the creative economy Stian Westlakersquos book with Jonathan Haskel Capitalism without Capital provides an excellent overview

Public Value How can it be measured managed and grown

7

Better metrics do not of themselves deliver better outcomes You canrsquot fatten a pig by weighing it But if you donrsquot have some means of weighing it you may find yourself unable to persuade others that itrsquos as fat as you believe

Many methods try to put a price on value The idea that the value to be gained from a new training programme can be directly compared with the value from a health screening programme or water conservation is immediately appealing to busy bureaucrats and ministers who have to justify spending money on one area over another as it is to foundations

So how can this be done Traditional economic valuation methods (based on utilitarian principles) which try to monetise public value usually draw either on what people say they would pay for a service or outcome (lsquostated preferencersquo) or on the choices people have made in related fields (lsquorevealed preferencersquo) These methods also try to adjust the estimated benefits of public services for quality of outcomes ndash for example comparing school exam results or the success of operations As they deliver commensurable estimates economic methods imply that the benefits of disparate outcomes can be compared and added up Other economic methods such as subjective wellbeing2 compare public policy actions by estimating the extra income people would need to achieve an equivalent gain in life satisfaction

Decision-making becomes a lot easier if any option can be turned into a comparable number and there have been many attempts to create such tools Governments would love to be able to use a single metric to compare spending on a scheme for housing the homeless primary education bridges and defence though none have come even remotely close to having such a tool Other attempts include CBA and in the social sector social return on investment (SROI) and more recently Y Analytics and the Impact Multiple of Money a single number now being promoted to adjust company valuations to take account of all their impacts

These are all well-intentioned But paying too much attention to monetary equivalence can lead to bad decisions and Irsquove become sceptical of moves to pretend that a single number can compare very different activities and outcomes These fall apart intellectually and practically for many reasons (I set out the arguments in more detail in this piece)

Some are reasons of principle (the plurality of approaches to valuation in different fields or spheres) some reasons fall on the boundaries of theory and practice (such as what discount rates to apply to different activities) and some are reasons of practicality (the experience that different methods used to assess value can generate wildly different numbers and often miss out what people turn out to value most (because people may not be willing to countenance trading off the things they care about most)3 As one recent book put it value lsquois neither an intrinsic quality of an object nor a reflection of a subjects preferences but rather something that is organized and brought into existence through mechanisms technologies and practices of valuationrsquo4 So our attention needs to focus on what these mechanisms are and whether they are fit for purpose and how they can be made useful rather than misleading

Public Value How can it be measured managed and grown

8

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society providing a bridge between the complex patterns of public demands and needs as expressed through political and other processes and the changing production systems that keep people healthy educated or safe

Cost-benefit analysis (CBA) has been the most widely used mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs)5 CBA has been subject to criticism for decades but continues to be used and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA

In philanthropy and civil society many methods are used some linked into standards of evidence of the kind Nesta has promoted These aim to capture data on impacts where possible to draw on longitudinal data sets and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1000 different methods in use) Often these attempt to provide versions of lsquobalanced scorecardsrsquo or outcome stars to map value in multiple dimensions

There are also the many applications of public sector welfare economics such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions Detailed methods have been in use for several decades to estimate the direct costs of an action (eg a drug treatment programme) the probability of it working and the likely impact on future crime rates hospital admissions or welfare payments

Analysis of this kind can be very powerful In the USA for example researchers identified what they called lsquomillion dollar blocksrsquo where the costs associated with criminals topped the million-dollar mark In principle good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime

These analyses can also be made highly sophisticated They can for example be adapted to reflect income distribution on the principle that an extra dollar or an extra unit of utility is worth more to a poor person than a rich one (since 2003 the UK Governmentrsquos Green Book has required all appraisals to include distributional effects) or time effects (on the principle that if growth continues the marginal value of an extra pound will be worth less in 50 yearsrsquo time simply because average incomes will be higher)

But all measurements of complex effects are inherently hard and they can be costly The benefits tend to become more dispersed over time affecting ever more agencies in ever more uncertain ways Social science is rarely robust enough to make hard predictions about what causes will lead to what effects ndash there are usually far too many variables involved Meanwhile standard cost benefit models apply discount rates to these gains usually based on prevailing commercial interest rates which renders a benefit in a generationrsquos time virtually worthless

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable But in many of the most important fields for government action or philanthropy ndash like childcare crime prevention or schooling ndash the public are divided over values as well as value For most people for example there is an

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 4: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

4

Public Value How can it be measured managed and grown

SummaryRecent decades have brought periodic waves of interest in finding better ways of mapping and measuring public and social value - outcomes institutions and services that are valued by the public but not easy to count in the way that the monetary value of cars or computers can be Governments and parliaments have sometimes tried to encourage this In the UK for example the Social Value Act explicitly encouraged broader notions of value as have recent revisions of the Treasury Green Book used to assess public spending choices

But these generally remain underdeveloped and lack influence compared to more traditional market valuation methods Within government there are still far more developed means for measuring investment in physical things - railways roads airports - than in intangibles like reducing social isolation improving health or the arts This almost certainly continues to skew public spending

Itrsquos obviously wrong to claim that you canrsquot manage what you canrsquot measure But the lack of well-established metrics for public value in all its forms makes it harder to compare alternatives and continues to bias public spending against prevention rather than cure against indirect benefits relative to direct ones and against intangibles relative to physical objects

This paper sets out alternatives It brings together views from Nesta on better ways of mapping and measuring public value It builds on work Nesta has done in many fields - from health and culture to public services - to find more rounded and realistic ways of capturing the many dimensions of value created by public action It is relevant to our work influencing governments and charities as well as to our own work as a funder since our status as a charity commits us to creating public benefit

Our aim in this work is to make value more transparent and more open to interrogation whether that concerns libraries bicycle lanes museums primary health services or training programmes for the unemployed We recognise that value may come from government action it can also be created by others in civil society and business And we recognise that value can often be complex whether in terms of who benefits or how it relates to values as well as more technical issues such as what discount rates to apply

But unless value is attended to explicitly we risk ending up with unhappy results This is why we have focused so much on how to both understand as well as create value through setting up What Works Centres and the Alliance for Useful Evidence through impact investment funds that use more precise metrics for social impact through projects in health that recognise what is important to people themselves through projects to measure the value of culture and through programmes like lsquoInnovate to Saversquo in Wales that aims to achieve cashable and other savings from innovations

5

Public Value How can it be measured managed and grown

This paper documents work in progress but we hope its concepts and applications will be useful It shows that

1 Despite the tensions and dilemmas public value - in all its forms - can be mapped and often measured in ways that are useful for shaping decisions

2 The best valuation methods avoid the twin errors of on the one hand believing that a single measure can adequately capture complex values and on the other believing that decisions should just be guided by hunch or intuition

3 Public value like value in markets - is not a static fact but arises from the interaction of changing options for supply and changing demands and public priorities

4 The greatest use of measures of public value comes not just from more detailed analysis but from the conversations and negotiations they prompt The key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

5 Public value methods should help governments decide on actions now that can deliver benefits in the long term whether on climate change early childhood or healthcare This requires new ways of managing public investment but also a much more sophisticated approach to discount rates

6 The debate about public value needs to be tied into much more energetic reform of public finance methods which need to become more transparent more focused on outcomes better suited to experiment and innovation and better shaped for public engagement If this doesnrsquot happen anachronistic methods for managing finance will crush imaginative attempts to map and measure public value

7 Valuation has limits in a democracy it should be possible for citizens or politicians to decide that they donrsquot want the state involved either in measuring or seeking to influence important areas of life including families and communities

This field is more of a craft than a science But itrsquos one with a growing range of tools that can in time evolve to become more reliable and rigorous We advocate more work to seek out value and where possible to measure it and use it to guide spending decisions and we believe that this will increasingly involve new data sources including public administrative data We hope that governments will commit themselves to using public value to guide their decisions and future budgets and to catalyse long-overdue reform to the processes of public finance

Public Value How can it be measured managed and grown

6

Background Searching for value By Geoff Mulgan

Until the 1990s international accounting conventions assumed that public sector productivity never improved the value of public provision was assumed to be equal to its cost Yet in a period of greater public pressure for results and value for money that position became untenable

For a time the main focus was on critique ndash showing the limitations of mainstream economics and methods like cost-benefit analysis and how inadequately they captured externalities and social benefits

But over the last 20 years serious attention has turned to finding better alternatives (though critiques can still pull in a big audience) Public bodies like the BBC commissioned detailed studies of the public value they were creating back in the 2000s and for a time the Cabinet Office promoted new methods of assessing public value These methods aim to measure not just what public interventions cost but also provide more rounded accounts of what value they create and just as important what value civil society can provide for public commissioners and purchasers There have been advances both in economic approaches ndash like Cost Benefit Analysis (CBA) and the UK Treasury Green Book ndash and non-economic ones that are mirrored in many countries around the world Ambitious ideas are also being developed for more precise measures of social and other goods1

Therersquos been a parallel push in civil society mainly driven by funders wanting better measures of social impact and social returns better measures for the impact in lsquoimpact investmentrsquo and better ways to compare the results of spending in different fields A plethora of organisations have been set up to help map and measure value and impact and ndash like Global Impact Investment Network ndash to establish some common standards for fields like clean energy or healthcare Others offer to measure social value

At least four distinct communities have been having very similar discussions about value though with surprisingly little cross-pollination statisticians attempting to measure wellbeing and outcomes of all kinds (for example brought together in the OECD Better Life Index) economists applying measurement tools to new fields such as carbon and natural capital philanthropists and NGOs interested in impact investment and activists using new measures as a tool for social change (for example measuring or revealing carbon footprints or social neglect)

Not too far from these there has also been an explosion of methods for measuring intangibles ndash the value of ideas knowledge creativity ndash many of which Nesta has been closely involved in Some measure innovation spending and the value it creates others the inputs and outputs of the creative economy Stian Westlakersquos book with Jonathan Haskel Capitalism without Capital provides an excellent overview

Public Value How can it be measured managed and grown

7

Better metrics do not of themselves deliver better outcomes You canrsquot fatten a pig by weighing it But if you donrsquot have some means of weighing it you may find yourself unable to persuade others that itrsquos as fat as you believe

Many methods try to put a price on value The idea that the value to be gained from a new training programme can be directly compared with the value from a health screening programme or water conservation is immediately appealing to busy bureaucrats and ministers who have to justify spending money on one area over another as it is to foundations

So how can this be done Traditional economic valuation methods (based on utilitarian principles) which try to monetise public value usually draw either on what people say they would pay for a service or outcome (lsquostated preferencersquo) or on the choices people have made in related fields (lsquorevealed preferencersquo) These methods also try to adjust the estimated benefits of public services for quality of outcomes ndash for example comparing school exam results or the success of operations As they deliver commensurable estimates economic methods imply that the benefits of disparate outcomes can be compared and added up Other economic methods such as subjective wellbeing2 compare public policy actions by estimating the extra income people would need to achieve an equivalent gain in life satisfaction

Decision-making becomes a lot easier if any option can be turned into a comparable number and there have been many attempts to create such tools Governments would love to be able to use a single metric to compare spending on a scheme for housing the homeless primary education bridges and defence though none have come even remotely close to having such a tool Other attempts include CBA and in the social sector social return on investment (SROI) and more recently Y Analytics and the Impact Multiple of Money a single number now being promoted to adjust company valuations to take account of all their impacts

These are all well-intentioned But paying too much attention to monetary equivalence can lead to bad decisions and Irsquove become sceptical of moves to pretend that a single number can compare very different activities and outcomes These fall apart intellectually and practically for many reasons (I set out the arguments in more detail in this piece)

Some are reasons of principle (the plurality of approaches to valuation in different fields or spheres) some reasons fall on the boundaries of theory and practice (such as what discount rates to apply to different activities) and some are reasons of practicality (the experience that different methods used to assess value can generate wildly different numbers and often miss out what people turn out to value most (because people may not be willing to countenance trading off the things they care about most)3 As one recent book put it value lsquois neither an intrinsic quality of an object nor a reflection of a subjects preferences but rather something that is organized and brought into existence through mechanisms technologies and practices of valuationrsquo4 So our attention needs to focus on what these mechanisms are and whether they are fit for purpose and how they can be made useful rather than misleading

Public Value How can it be measured managed and grown

8

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society providing a bridge between the complex patterns of public demands and needs as expressed through political and other processes and the changing production systems that keep people healthy educated or safe

Cost-benefit analysis (CBA) has been the most widely used mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs)5 CBA has been subject to criticism for decades but continues to be used and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA

In philanthropy and civil society many methods are used some linked into standards of evidence of the kind Nesta has promoted These aim to capture data on impacts where possible to draw on longitudinal data sets and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1000 different methods in use) Often these attempt to provide versions of lsquobalanced scorecardsrsquo or outcome stars to map value in multiple dimensions

There are also the many applications of public sector welfare economics such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions Detailed methods have been in use for several decades to estimate the direct costs of an action (eg a drug treatment programme) the probability of it working and the likely impact on future crime rates hospital admissions or welfare payments

Analysis of this kind can be very powerful In the USA for example researchers identified what they called lsquomillion dollar blocksrsquo where the costs associated with criminals topped the million-dollar mark In principle good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime

These analyses can also be made highly sophisticated They can for example be adapted to reflect income distribution on the principle that an extra dollar or an extra unit of utility is worth more to a poor person than a rich one (since 2003 the UK Governmentrsquos Green Book has required all appraisals to include distributional effects) or time effects (on the principle that if growth continues the marginal value of an extra pound will be worth less in 50 yearsrsquo time simply because average incomes will be higher)

But all measurements of complex effects are inherently hard and they can be costly The benefits tend to become more dispersed over time affecting ever more agencies in ever more uncertain ways Social science is rarely robust enough to make hard predictions about what causes will lead to what effects ndash there are usually far too many variables involved Meanwhile standard cost benefit models apply discount rates to these gains usually based on prevailing commercial interest rates which renders a benefit in a generationrsquos time virtually worthless

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable But in many of the most important fields for government action or philanthropy ndash like childcare crime prevention or schooling ndash the public are divided over values as well as value For most people for example there is an

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 5: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

5

Public Value How can it be measured managed and grown

This paper documents work in progress but we hope its concepts and applications will be useful It shows that

1 Despite the tensions and dilemmas public value - in all its forms - can be mapped and often measured in ways that are useful for shaping decisions

2 The best valuation methods avoid the twin errors of on the one hand believing that a single measure can adequately capture complex values and on the other believing that decisions should just be guided by hunch or intuition

3 Public value like value in markets - is not a static fact but arises from the interaction of changing options for supply and changing demands and public priorities

4 The greatest use of measures of public value comes not just from more detailed analysis but from the conversations and negotiations they prompt The key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

5 Public value methods should help governments decide on actions now that can deliver benefits in the long term whether on climate change early childhood or healthcare This requires new ways of managing public investment but also a much more sophisticated approach to discount rates

6 The debate about public value needs to be tied into much more energetic reform of public finance methods which need to become more transparent more focused on outcomes better suited to experiment and innovation and better shaped for public engagement If this doesnrsquot happen anachronistic methods for managing finance will crush imaginative attempts to map and measure public value

7 Valuation has limits in a democracy it should be possible for citizens or politicians to decide that they donrsquot want the state involved either in measuring or seeking to influence important areas of life including families and communities

This field is more of a craft than a science But itrsquos one with a growing range of tools that can in time evolve to become more reliable and rigorous We advocate more work to seek out value and where possible to measure it and use it to guide spending decisions and we believe that this will increasingly involve new data sources including public administrative data We hope that governments will commit themselves to using public value to guide their decisions and future budgets and to catalyse long-overdue reform to the processes of public finance

Public Value How can it be measured managed and grown

6

Background Searching for value By Geoff Mulgan

Until the 1990s international accounting conventions assumed that public sector productivity never improved the value of public provision was assumed to be equal to its cost Yet in a period of greater public pressure for results and value for money that position became untenable

For a time the main focus was on critique ndash showing the limitations of mainstream economics and methods like cost-benefit analysis and how inadequately they captured externalities and social benefits

But over the last 20 years serious attention has turned to finding better alternatives (though critiques can still pull in a big audience) Public bodies like the BBC commissioned detailed studies of the public value they were creating back in the 2000s and for a time the Cabinet Office promoted new methods of assessing public value These methods aim to measure not just what public interventions cost but also provide more rounded accounts of what value they create and just as important what value civil society can provide for public commissioners and purchasers There have been advances both in economic approaches ndash like Cost Benefit Analysis (CBA) and the UK Treasury Green Book ndash and non-economic ones that are mirrored in many countries around the world Ambitious ideas are also being developed for more precise measures of social and other goods1

Therersquos been a parallel push in civil society mainly driven by funders wanting better measures of social impact and social returns better measures for the impact in lsquoimpact investmentrsquo and better ways to compare the results of spending in different fields A plethora of organisations have been set up to help map and measure value and impact and ndash like Global Impact Investment Network ndash to establish some common standards for fields like clean energy or healthcare Others offer to measure social value

At least four distinct communities have been having very similar discussions about value though with surprisingly little cross-pollination statisticians attempting to measure wellbeing and outcomes of all kinds (for example brought together in the OECD Better Life Index) economists applying measurement tools to new fields such as carbon and natural capital philanthropists and NGOs interested in impact investment and activists using new measures as a tool for social change (for example measuring or revealing carbon footprints or social neglect)

Not too far from these there has also been an explosion of methods for measuring intangibles ndash the value of ideas knowledge creativity ndash many of which Nesta has been closely involved in Some measure innovation spending and the value it creates others the inputs and outputs of the creative economy Stian Westlakersquos book with Jonathan Haskel Capitalism without Capital provides an excellent overview

Public Value How can it be measured managed and grown

7

Better metrics do not of themselves deliver better outcomes You canrsquot fatten a pig by weighing it But if you donrsquot have some means of weighing it you may find yourself unable to persuade others that itrsquos as fat as you believe

Many methods try to put a price on value The idea that the value to be gained from a new training programme can be directly compared with the value from a health screening programme or water conservation is immediately appealing to busy bureaucrats and ministers who have to justify spending money on one area over another as it is to foundations

So how can this be done Traditional economic valuation methods (based on utilitarian principles) which try to monetise public value usually draw either on what people say they would pay for a service or outcome (lsquostated preferencersquo) or on the choices people have made in related fields (lsquorevealed preferencersquo) These methods also try to adjust the estimated benefits of public services for quality of outcomes ndash for example comparing school exam results or the success of operations As they deliver commensurable estimates economic methods imply that the benefits of disparate outcomes can be compared and added up Other economic methods such as subjective wellbeing2 compare public policy actions by estimating the extra income people would need to achieve an equivalent gain in life satisfaction

Decision-making becomes a lot easier if any option can be turned into a comparable number and there have been many attempts to create such tools Governments would love to be able to use a single metric to compare spending on a scheme for housing the homeless primary education bridges and defence though none have come even remotely close to having such a tool Other attempts include CBA and in the social sector social return on investment (SROI) and more recently Y Analytics and the Impact Multiple of Money a single number now being promoted to adjust company valuations to take account of all their impacts

These are all well-intentioned But paying too much attention to monetary equivalence can lead to bad decisions and Irsquove become sceptical of moves to pretend that a single number can compare very different activities and outcomes These fall apart intellectually and practically for many reasons (I set out the arguments in more detail in this piece)

Some are reasons of principle (the plurality of approaches to valuation in different fields or spheres) some reasons fall on the boundaries of theory and practice (such as what discount rates to apply to different activities) and some are reasons of practicality (the experience that different methods used to assess value can generate wildly different numbers and often miss out what people turn out to value most (because people may not be willing to countenance trading off the things they care about most)3 As one recent book put it value lsquois neither an intrinsic quality of an object nor a reflection of a subjects preferences but rather something that is organized and brought into existence through mechanisms technologies and practices of valuationrsquo4 So our attention needs to focus on what these mechanisms are and whether they are fit for purpose and how they can be made useful rather than misleading

Public Value How can it be measured managed and grown

8

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society providing a bridge between the complex patterns of public demands and needs as expressed through political and other processes and the changing production systems that keep people healthy educated or safe

Cost-benefit analysis (CBA) has been the most widely used mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs)5 CBA has been subject to criticism for decades but continues to be used and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA

In philanthropy and civil society many methods are used some linked into standards of evidence of the kind Nesta has promoted These aim to capture data on impacts where possible to draw on longitudinal data sets and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1000 different methods in use) Often these attempt to provide versions of lsquobalanced scorecardsrsquo or outcome stars to map value in multiple dimensions

There are also the many applications of public sector welfare economics such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions Detailed methods have been in use for several decades to estimate the direct costs of an action (eg a drug treatment programme) the probability of it working and the likely impact on future crime rates hospital admissions or welfare payments

Analysis of this kind can be very powerful In the USA for example researchers identified what they called lsquomillion dollar blocksrsquo where the costs associated with criminals topped the million-dollar mark In principle good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime

These analyses can also be made highly sophisticated They can for example be adapted to reflect income distribution on the principle that an extra dollar or an extra unit of utility is worth more to a poor person than a rich one (since 2003 the UK Governmentrsquos Green Book has required all appraisals to include distributional effects) or time effects (on the principle that if growth continues the marginal value of an extra pound will be worth less in 50 yearsrsquo time simply because average incomes will be higher)

But all measurements of complex effects are inherently hard and they can be costly The benefits tend to become more dispersed over time affecting ever more agencies in ever more uncertain ways Social science is rarely robust enough to make hard predictions about what causes will lead to what effects ndash there are usually far too many variables involved Meanwhile standard cost benefit models apply discount rates to these gains usually based on prevailing commercial interest rates which renders a benefit in a generationrsquos time virtually worthless

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable But in many of the most important fields for government action or philanthropy ndash like childcare crime prevention or schooling ndash the public are divided over values as well as value For most people for example there is an

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 6: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

6

Background Searching for value By Geoff Mulgan

Until the 1990s international accounting conventions assumed that public sector productivity never improved the value of public provision was assumed to be equal to its cost Yet in a period of greater public pressure for results and value for money that position became untenable

For a time the main focus was on critique ndash showing the limitations of mainstream economics and methods like cost-benefit analysis and how inadequately they captured externalities and social benefits

But over the last 20 years serious attention has turned to finding better alternatives (though critiques can still pull in a big audience) Public bodies like the BBC commissioned detailed studies of the public value they were creating back in the 2000s and for a time the Cabinet Office promoted new methods of assessing public value These methods aim to measure not just what public interventions cost but also provide more rounded accounts of what value they create and just as important what value civil society can provide for public commissioners and purchasers There have been advances both in economic approaches ndash like Cost Benefit Analysis (CBA) and the UK Treasury Green Book ndash and non-economic ones that are mirrored in many countries around the world Ambitious ideas are also being developed for more precise measures of social and other goods1

Therersquos been a parallel push in civil society mainly driven by funders wanting better measures of social impact and social returns better measures for the impact in lsquoimpact investmentrsquo and better ways to compare the results of spending in different fields A plethora of organisations have been set up to help map and measure value and impact and ndash like Global Impact Investment Network ndash to establish some common standards for fields like clean energy or healthcare Others offer to measure social value

At least four distinct communities have been having very similar discussions about value though with surprisingly little cross-pollination statisticians attempting to measure wellbeing and outcomes of all kinds (for example brought together in the OECD Better Life Index) economists applying measurement tools to new fields such as carbon and natural capital philanthropists and NGOs interested in impact investment and activists using new measures as a tool for social change (for example measuring or revealing carbon footprints or social neglect)

Not too far from these there has also been an explosion of methods for measuring intangibles ndash the value of ideas knowledge creativity ndash many of which Nesta has been closely involved in Some measure innovation spending and the value it creates others the inputs and outputs of the creative economy Stian Westlakersquos book with Jonathan Haskel Capitalism without Capital provides an excellent overview

Public Value How can it be measured managed and grown

7

Better metrics do not of themselves deliver better outcomes You canrsquot fatten a pig by weighing it But if you donrsquot have some means of weighing it you may find yourself unable to persuade others that itrsquos as fat as you believe

Many methods try to put a price on value The idea that the value to be gained from a new training programme can be directly compared with the value from a health screening programme or water conservation is immediately appealing to busy bureaucrats and ministers who have to justify spending money on one area over another as it is to foundations

So how can this be done Traditional economic valuation methods (based on utilitarian principles) which try to monetise public value usually draw either on what people say they would pay for a service or outcome (lsquostated preferencersquo) or on the choices people have made in related fields (lsquorevealed preferencersquo) These methods also try to adjust the estimated benefits of public services for quality of outcomes ndash for example comparing school exam results or the success of operations As they deliver commensurable estimates economic methods imply that the benefits of disparate outcomes can be compared and added up Other economic methods such as subjective wellbeing2 compare public policy actions by estimating the extra income people would need to achieve an equivalent gain in life satisfaction

Decision-making becomes a lot easier if any option can be turned into a comparable number and there have been many attempts to create such tools Governments would love to be able to use a single metric to compare spending on a scheme for housing the homeless primary education bridges and defence though none have come even remotely close to having such a tool Other attempts include CBA and in the social sector social return on investment (SROI) and more recently Y Analytics and the Impact Multiple of Money a single number now being promoted to adjust company valuations to take account of all their impacts

These are all well-intentioned But paying too much attention to monetary equivalence can lead to bad decisions and Irsquove become sceptical of moves to pretend that a single number can compare very different activities and outcomes These fall apart intellectually and practically for many reasons (I set out the arguments in more detail in this piece)

Some are reasons of principle (the plurality of approaches to valuation in different fields or spheres) some reasons fall on the boundaries of theory and practice (such as what discount rates to apply to different activities) and some are reasons of practicality (the experience that different methods used to assess value can generate wildly different numbers and often miss out what people turn out to value most (because people may not be willing to countenance trading off the things they care about most)3 As one recent book put it value lsquois neither an intrinsic quality of an object nor a reflection of a subjects preferences but rather something that is organized and brought into existence through mechanisms technologies and practices of valuationrsquo4 So our attention needs to focus on what these mechanisms are and whether they are fit for purpose and how they can be made useful rather than misleading

Public Value How can it be measured managed and grown

8

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society providing a bridge between the complex patterns of public demands and needs as expressed through political and other processes and the changing production systems that keep people healthy educated or safe

Cost-benefit analysis (CBA) has been the most widely used mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs)5 CBA has been subject to criticism for decades but continues to be used and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA

In philanthropy and civil society many methods are used some linked into standards of evidence of the kind Nesta has promoted These aim to capture data on impacts where possible to draw on longitudinal data sets and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1000 different methods in use) Often these attempt to provide versions of lsquobalanced scorecardsrsquo or outcome stars to map value in multiple dimensions

There are also the many applications of public sector welfare economics such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions Detailed methods have been in use for several decades to estimate the direct costs of an action (eg a drug treatment programme) the probability of it working and the likely impact on future crime rates hospital admissions or welfare payments

Analysis of this kind can be very powerful In the USA for example researchers identified what they called lsquomillion dollar blocksrsquo where the costs associated with criminals topped the million-dollar mark In principle good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime

These analyses can also be made highly sophisticated They can for example be adapted to reflect income distribution on the principle that an extra dollar or an extra unit of utility is worth more to a poor person than a rich one (since 2003 the UK Governmentrsquos Green Book has required all appraisals to include distributional effects) or time effects (on the principle that if growth continues the marginal value of an extra pound will be worth less in 50 yearsrsquo time simply because average incomes will be higher)

But all measurements of complex effects are inherently hard and they can be costly The benefits tend to become more dispersed over time affecting ever more agencies in ever more uncertain ways Social science is rarely robust enough to make hard predictions about what causes will lead to what effects ndash there are usually far too many variables involved Meanwhile standard cost benefit models apply discount rates to these gains usually based on prevailing commercial interest rates which renders a benefit in a generationrsquos time virtually worthless

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable But in many of the most important fields for government action or philanthropy ndash like childcare crime prevention or schooling ndash the public are divided over values as well as value For most people for example there is an

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 7: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

7

Better metrics do not of themselves deliver better outcomes You canrsquot fatten a pig by weighing it But if you donrsquot have some means of weighing it you may find yourself unable to persuade others that itrsquos as fat as you believe

Many methods try to put a price on value The idea that the value to be gained from a new training programme can be directly compared with the value from a health screening programme or water conservation is immediately appealing to busy bureaucrats and ministers who have to justify spending money on one area over another as it is to foundations

So how can this be done Traditional economic valuation methods (based on utilitarian principles) which try to monetise public value usually draw either on what people say they would pay for a service or outcome (lsquostated preferencersquo) or on the choices people have made in related fields (lsquorevealed preferencersquo) These methods also try to adjust the estimated benefits of public services for quality of outcomes ndash for example comparing school exam results or the success of operations As they deliver commensurable estimates economic methods imply that the benefits of disparate outcomes can be compared and added up Other economic methods such as subjective wellbeing2 compare public policy actions by estimating the extra income people would need to achieve an equivalent gain in life satisfaction

Decision-making becomes a lot easier if any option can be turned into a comparable number and there have been many attempts to create such tools Governments would love to be able to use a single metric to compare spending on a scheme for housing the homeless primary education bridges and defence though none have come even remotely close to having such a tool Other attempts include CBA and in the social sector social return on investment (SROI) and more recently Y Analytics and the Impact Multiple of Money a single number now being promoted to adjust company valuations to take account of all their impacts

These are all well-intentioned But paying too much attention to monetary equivalence can lead to bad decisions and Irsquove become sceptical of moves to pretend that a single number can compare very different activities and outcomes These fall apart intellectually and practically for many reasons (I set out the arguments in more detail in this piece)

Some are reasons of principle (the plurality of approaches to valuation in different fields or spheres) some reasons fall on the boundaries of theory and practice (such as what discount rates to apply to different activities) and some are reasons of practicality (the experience that different methods used to assess value can generate wildly different numbers and often miss out what people turn out to value most (because people may not be willing to countenance trading off the things they care about most)3 As one recent book put it value lsquois neither an intrinsic quality of an object nor a reflection of a subjects preferences but rather something that is organized and brought into existence through mechanisms technologies and practices of valuationrsquo4 So our attention needs to focus on what these mechanisms are and whether they are fit for purpose and how they can be made useful rather than misleading

Public Value How can it be measured managed and grown

8

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society providing a bridge between the complex patterns of public demands and needs as expressed through political and other processes and the changing production systems that keep people healthy educated or safe

Cost-benefit analysis (CBA) has been the most widely used mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs)5 CBA has been subject to criticism for decades but continues to be used and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA

In philanthropy and civil society many methods are used some linked into standards of evidence of the kind Nesta has promoted These aim to capture data on impacts where possible to draw on longitudinal data sets and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1000 different methods in use) Often these attempt to provide versions of lsquobalanced scorecardsrsquo or outcome stars to map value in multiple dimensions

There are also the many applications of public sector welfare economics such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions Detailed methods have been in use for several decades to estimate the direct costs of an action (eg a drug treatment programme) the probability of it working and the likely impact on future crime rates hospital admissions or welfare payments

Analysis of this kind can be very powerful In the USA for example researchers identified what they called lsquomillion dollar blocksrsquo where the costs associated with criminals topped the million-dollar mark In principle good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime

These analyses can also be made highly sophisticated They can for example be adapted to reflect income distribution on the principle that an extra dollar or an extra unit of utility is worth more to a poor person than a rich one (since 2003 the UK Governmentrsquos Green Book has required all appraisals to include distributional effects) or time effects (on the principle that if growth continues the marginal value of an extra pound will be worth less in 50 yearsrsquo time simply because average incomes will be higher)

But all measurements of complex effects are inherently hard and they can be costly The benefits tend to become more dispersed over time affecting ever more agencies in ever more uncertain ways Social science is rarely robust enough to make hard predictions about what causes will lead to what effects ndash there are usually far too many variables involved Meanwhile standard cost benefit models apply discount rates to these gains usually based on prevailing commercial interest rates which renders a benefit in a generationrsquos time virtually worthless

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable But in many of the most important fields for government action or philanthropy ndash like childcare crime prevention or schooling ndash the public are divided over values as well as value For most people for example there is an

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 8: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

8

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society providing a bridge between the complex patterns of public demands and needs as expressed through political and other processes and the changing production systems that keep people healthy educated or safe

Cost-benefit analysis (CBA) has been the most widely used mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs)5 CBA has been subject to criticism for decades but continues to be used and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA

In philanthropy and civil society many methods are used some linked into standards of evidence of the kind Nesta has promoted These aim to capture data on impacts where possible to draw on longitudinal data sets and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1000 different methods in use) Often these attempt to provide versions of lsquobalanced scorecardsrsquo or outcome stars to map value in multiple dimensions

There are also the many applications of public sector welfare economics such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions Detailed methods have been in use for several decades to estimate the direct costs of an action (eg a drug treatment programme) the probability of it working and the likely impact on future crime rates hospital admissions or welfare payments

Analysis of this kind can be very powerful In the USA for example researchers identified what they called lsquomillion dollar blocksrsquo where the costs associated with criminals topped the million-dollar mark In principle good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime

These analyses can also be made highly sophisticated They can for example be adapted to reflect income distribution on the principle that an extra dollar or an extra unit of utility is worth more to a poor person than a rich one (since 2003 the UK Governmentrsquos Green Book has required all appraisals to include distributional effects) or time effects (on the principle that if growth continues the marginal value of an extra pound will be worth less in 50 yearsrsquo time simply because average incomes will be higher)

But all measurements of complex effects are inherently hard and they can be costly The benefits tend to become more dispersed over time affecting ever more agencies in ever more uncertain ways Social science is rarely robust enough to make hard predictions about what causes will lead to what effects ndash there are usually far too many variables involved Meanwhile standard cost benefit models apply discount rates to these gains usually based on prevailing commercial interest rates which renders a benefit in a generationrsquos time virtually worthless

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable But in many of the most important fields for government action or philanthropy ndash like childcare crime prevention or schooling ndash the public are divided over values as well as value For most people for example there is an

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 9: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

9

intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison6 This is why the economic models for thinking about public goods and externalities though informative are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics

Many valuation methods also fail to take account of what matters to beneficiaries This is particularly the case for childrenrsquos services and care Too often children are powerless and voiceless and their concerns are excluded from the clever metrics designed to capture impact

As a result the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions

Box 1 The Social Value Act (2013)

The aim of the Social Value Act was to shape the procurement approach and design of public services in England The Act encourages commissioners to take a value for money approach not the lowest cost approach Nonetheless the Act does not specify how procurement models should incorporate social value or how social value should be measured At the Social Value Hub one of the four key areas within the actrsquos programme of work different reports suggest SROI can be a useful tool to measure social value However there are no reports indicating how many organisations are using SROI

Implementing the Social Value Act

Lord Youngrsquos two-year review of the act highlighted the issues caused by the lack of a fully developed measurement of social value The review pointed out that despite many organisations seeing the potential of the Act it was difficult for public bodies to differentiate the additional social value offered by one bidder over another Youngrsquos (2015) review also acknowledges that fewer organisations had fully developed a strategy or policy in place and therefore there were inconsistent practices According to the report Procuring for Good (2016) 33 per cent of councils make use of the Act but only 24 per cent had a social value policy in place Healthy Commissioning (2017) found that only 13 per cent of Clinical Commissioning Groups had committed

evidenced and made active use of the Act whereas 57 per cent of CCGs included the term social value in their procurement processes Power to Change reporting on the Act also found that it made little practical difference to how councils commissioned

Future recommendations

UK Government announced in early 2018 an extension of the act social value will be an explicit requirement in central government contracts to the private and third sectors (not local government or other public sector commissioners) This extension does not seem to include a fully developed measurement of social value Nevertheless there are other relevant initiatives in this area The British Standards Institute is developing a standard for social value with representatives from across sectors ISO 20400 the first international guidance standard on sustainable procurement has incorporated social environmental and economic concerns into the procurement activity and supply chains Scotland and Wales have also passed similar initiatives In 2014 Scotland passed the Procurement Reform Act and Wales the Community Benefits Guidance The Scottish model of procurement includes public services and goods focusing on a business-friendly and socially responsible procurement approach The Welsh proposal highlights the idea of sustainable development as the central organising principle

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 10: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

10

Services outcomes and justice

Many things that governments and civil society do are valuable to the public but they roughly fall into three main categories

The first is the value provided by services ndash like well-maintained roads or hospitals Services can be relatively easy to analyse in terms of value In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service ndash for example the evening opening of a library or retaining a very local Post Office

The second category of value is outcomes ndash like lower crime or security from invasion Again through democratic argument or through surveys people can place a rough value on these One of the outcomes they say they often value highly is equity Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending Often outcomes and services align but in many cases they point in different directions (eg more police on the street experienced as an improvement in services even when it doesnrsquot result in lower crime)

A third category of value is trust in its widest sense which includes whether the work of government is seen as just and fair or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities)7 This is influenced partly by peoplersquos satisfaction with public services alongside perceived integrity reliability and responsiveness of the government Its flow-on effects matter too with lsquoperceived quality of institutionsrsquo in turn influencing peoplersquos trust in others8 Our approach has been to encourage an active search for value ndash using the best available tools for mapping and measurement ndash while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round In what follows we set out a series of case studies that show how different fields are grappling with these issues primarily focusing on work we are directly involved in

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 11: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

11

Value in the real world Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers

1 The What Works movement Bringing more rigorous views of impact and value into public services By Jonathan Breckon and Mariola Tarrega

The UKrsquos What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta they seek to marshall the best available evidence to make smarter policy practice and spending decisions

The current crop of What Works Centres range from big organisations like the pound125 million Education Endowment Foundation to diverse consortiums like the What Works Centre for Childrenrsquos Social Care hosted by Nesta with partners from academia business and the third sector Although most Centres receive public funding all are independent of government either established as charities or based in universities or armrsquos-length bodies

By using easy-to-understand quantifiable values they aid decision-making by pointing out what has worked before ndash according to international evaluations and research For instance the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below) Other centres compare measures in areas like wellbeing local economic growth quality of life reduced crime and childrenrsquos social care The hope is that by bundling up measures of value ndash particularly on effectiveness ndash these centres will save government money According to the UK Governmentrsquos 2012 Civil Service Reform Plan the What Works Centres

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 12: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

12

Public Value How can it be measured managed and grown

National Institute for Health and Care Excellence (NICE)

Education Endowment Foundation

What Works Centre for Crime Reduction

What Works Wellbeing

Early Intervention Foundation

Centre for Ageing Better

What Works Scotland

(affiliated)

Wales Centre for Public Policy

(affiliated)

Formerly Public Policy Institute for Wales

What Works Centre for Childrenrsquos Social Care

(affiliated)

1999 2011 2013 2014 2015 2017

What Works Centre for Local Economic Growth

What Works Centres timeline

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 13: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

13

helliptest and trial approaches and assess what works in major social policy areas so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money

HM Government 2012 p17

What Works Centres point to how to save money ndash by cutting things that donrsquot work9 But they also show how you can save money in other ways spending taxpayersrsquo money on successful things that are also cheaper than other options For instance reading comprehension strategies are both cheap and effective in schools according to extensive evidence Other successful approaches are just too expensive one-to-one mentoring for instance is reasonably effective but uses up too much time and money School budgets just canrsquot stretch that far

More controversially organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments because their value fails to justify their costs NICE ndash created in 1999 under the Blair government to end the lsquopostcode lotteryrsquo of availability and quality of NHS treatments ndash decides whether a clinical intervention or technology is merited Their evidence-based recommendations can however be hard for the public to stomach For instance guidance issued by NICE in 2005 caused a lsquofirestorm of controversyrsquo by banning drugs for Alzheimerrsquos known as cholinesterase inhibitors10 While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimerrsquos patients it doubted whether the size of the benefit was worth the hefty cost of the drugs However there were no other drugs available to patients at the time and NICE received over 8000 complaints11 Such controversies do not go unnoticed in other countries In the US Obamarsquos Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care12

Box 2 US Congress legislates against QALYs

The Affordable Care Act legislated against the use QALYs saying that lsquoThe Secretary shall not utilize such an adjusted life year (ie QALYS) (or such a similar measure) as a threshold to determine coverage reimbursement or incentive programsrsquo This might reflect the US aversion to lsquobig governmentrsquo and so-called lsquodeath panelsrsquo Nevertheless a ban on such valuation techniques has its own ethical dilemmas As the authors of an article in the prestigious New England Medical Journal pointed out

Taken literally it means that spending resources to extend by a month the life of a 100-year-old person who is in a vegetative state cannot be valued differently from spending resources to extend the life of a child by many healthy years Though the ACA may be seeking to avert discrimination it instead helps to perpetuate the current system of implicit rationing and hidden biases13

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 14: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

14

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres For the others there is a more mixed record of how they assess public value Only half of them have any clear way of comparing values ndash such as through cost-benefit analysis cost-effectiveness or cost-utility ratios (basically a medical variation of cost-effectiveness) Some like the What Works Centre for Crime Reduction just recognise costs but donrsquot compare it to value Yet others like the Centre for Ageing Better have no approach to value at all although they do have plans to use more economic measures in the future

Generally the Centres have not picked one common currency of cost and value Instead they have grown their measurements organically some would say haphazardly with no rigid fixed model (see Annex comparing the ten What Works Centres) This is understandable as the sectors they cover are so diverse covering clinics foster homes classrooms and more Shoehorning everything into one uber-measure would not win friends for the What Works Centres What matters to a nurse may be very different from a teacher But as will be argued below it can be confusing having ten different What Works Centres with ten different ways of thinking about value

Valuing children in care

In some sectors there is almost no consensus on how to measure what counts How for instance can you understand whether the care system is helping children in foster care or childrenrsquos homes Itrsquos not enough to look at these childrenrsquos outcomes in isolation ndash such as through their GCSE results or whether they go to university That approach doesnrsquot take into account the legacy of trauma abuse or neglect they may have suffered long before they came into care According to Louise Bazalgette Evidence Lead for the What Works Centre for Childrenrsquos Social Care incubated at Nesta

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because theyrsquore working through their trauma as well as separation from their families (eg going missing getting angry with carers or damaging property) That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance making you want to move that child somewhere else But what if this is what that child always does when they arrive in a new placement to test the commitment of the carers and see if they can really trust them Thatrsquos how some children end up having 25 placements in seven years In some childrenrsquos minds if they break down another placement maybe theyrsquoll be allowed to go home We must avoid these sorts of reductive judgements when thinking of children in care

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 15: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

15

If we want to know whether foster care or childrenrsquos homes are helping then a good start is to ask the children themselves as Coram Voicersquos Bright Spots project is doing ndash still a rare approach to measuring the quality of childrenrsquos care Where childrenrsquos feedback is sought in most cases it still isnrsquot being gathered and analysed systematically

We also need to listen to the childrenrsquos carers and social workers and provide them with the support they need to do what is sometimes a very demanding job The What Works Centre for Childrenrsquos Social Care has to find meaningful ways of understanding what makes for great care for children That means measuring the things that really make a difference ndash the quality and stability of childrenrsquos relationships their resilience and wellbeing getting a job ndash so that we can see which interventions support these kinds of outcomes ndash rather than just measuring the things that are easy to count

How to collect trustworthy data

The example above on Childrenrsquos Social Care alludes to the difficulty of getting good data from people when subjective views are important Sometimes itrsquos easier to collect independent figures such as exam results in schools or numbers of medical diagnoses But when quality of life comes into the equation then people need to be asked how they feel

For NICE their preferred tool is the EQ-5D questionnaire It asks patients about pain discomfort self-care anxiety and other factors that created 243 unique combinations of health14 For the What Works Centre for Wellbeing subjective views of life quality is their core business They have opted for questions on Life Satisfaction such as answering lsquooverall how satisfied are you with your life nowadaysrsquo Life Satisfaction is a lsquocommon currencyrsquo for subjective wellbeing15

But can you trust what people tell you Perceptions may not match reality For instance results from the British Crime Survey consistently show that peoplersquos perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim This is important when thinking about subjective wellbeing What is more important the fear of crime or the actual act of crime It would be misleading to wholly dismiss peoplersquos fears perceptions matter

One approach used by social scientists is to use lsquomixed methodsrsquo merging qualitative and quantitative evidence In other words splice the subjective views from say interviews with insights grounded in hard numbers Itrsquos no methodological panacea If qualitative evidence directly contradicts the quantitative ndash as in the crime example above ndash then there is not much you can do But using mixed methods is a decent compromise And there are worthy examples of attempts to bring the two together Nestarsquos pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses To probe the numbers they also interviewed people to check lsquoinside the bonnetrsquo of what was going on The researchers called this an lsquoRCT+rsquo approach mixing quantitative causal evaluation with qualitative and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months They did not)

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 16: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

16

Nestarsquos standards of evidence16

Nesta provides a framework for thinking about how much you can trust evidence It lays out five levels of evidence of impact ranging from early-stage Theory of Change (Level 1) to robust replicable evidence (Level 5) The framework supports cost-benefit analysis (at Level 4) As well as being used to assess Nestarsquos social investment funds since then the Standards of Evidence have been used in other parts of the organisation such as Centre for Social Action Innovation Fund a partnership between Nesta and the Cabinet Office Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3 The comparison group approach used did not indicate direct correlation between their intervention and their initial aims However there was enough evidence to demonstrate the long-term potential of continuing their programme

Level 5You have manuals systems and procedures to ensure consistent replication and positive impact

Level 4You have one + independent replication evaluations that confirms these conclusions

Level 3You can demonstrate causality using a control or comparison group

Level 2You capture data that shows positive change but you cannot confirm you caused this

Level 1You can describe what you do and why it matters logically coherently and convincingly

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 17: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

17

Citizens measuring creating and evaluating evidence

When measuring value itrsquos a wise move to use credible evidence tried-and-tested methods mix quantitative and qualitative and use frameworks such as the Nesta standards of evidence But the views of the public at large ndash the voters and taxpayers ndash may differ from those close to the project There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes To help get round this we mentioned before that NICE runs a survey of 3000 members of the public to check their views on Quality of Life Essentially itrsquos a survey of the gut instincts of people although made in the artificial environment of filling in a questionnaire A more sophisticated technique is to use deliberative lsquomini-publicsrsquo such as Citizen Juries or Consensus Conferences17 For example the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3) Like a legal jury they need to weigh up the evidence spend time really understanding the issue discuss trade-offs and come to some final conclusions The Citizen Councilrsquos recommendations are incorporated into lsquoSocial Value Judgementsrsquo and where appropriate into NICErsquos methodology on guidance for healthcare Citizen or user participation is a core part of the Scottish public sector ethos18 For instance for What Works Scotland involving the public in its work is essential Co-production is the guiding principle for much of its work such as the Scottish Participatory Budgetary Evaluation Toolkit The views and contributions of citizens and others are just as important as research evidence These initiatives require organisations to transfer part of their decision-making power to users andor citizens They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process19 Finally organisations need to be open to enact changes promoted by these initiatives ie adopting new evaluation measures

Box 3 lsquoScience is not enoughrsquo ndash NICErsquos public forum for social value judgements

NICErsquos Citizens Council is a unique example of an established and semi-permanent mini public integrated into a decision-making body It was established in 2002

The Citizens Council of 30 members meets once a year for two to three days The Council was set up to ensure that the perspectives of the public are included in the processes it uses to develop clinical guidance It explicitly excludes anybody involved already in health such as those employed by the NHS or patient advocacy groups

Working on the premise that lsquoscience is not enoughrsquo the Council aims to ensure that healthcare reflects the social values of the public20 The Councilrsquos meetings and reports focus

specifically on issues where social value judgments must be made and the moral and ethical issues that NICE should take into account in its guidance and methodology Its conclusions are incorporated into NICErsquos Social Value Judgements document

Topics the Citizens Council has looked at include what are the societal values that need to be considered when making decisions about trade-offs between equity and efficiency What are the social values that should inform discounting ndash the benefits and costs of healthcare that accrue in the future or what values should be used for social care ndash as compared with traditional medical health care]

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 18: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

18

Differing time horizons

Another important way to think about value is over different time scales The What Works Centres all handle timescales differently

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques The EEF values do not reflect benefits in the future such as employment and welfare ndash they only focus on educational attainment now in the school The danger with this approach is that it ignores a whole raft of other longer-term social emotional and societal benefits For instance pre-school support for kids such as Sure Start programmes can benefit children and families for years ndash and even decades later But for the EEF the immediate cost is prohibitively high ndash particularly when the evidence shows that the educational benefits are marginal21 However these costs may be worth it if it means a disadvantaged pupil goes on to have a better life ndash and avoid poverty joblessness crime misery and dysfunctional family life all of which are costly to the state

Box 4 Education Endowment Foundation Teaching and Learning Toolkit

For the Education Endowment Foundation value is narrowly defined as educational attainment Other outcomes are not used in their toolkits such as wellbeing character or social equity This approach is not without difficulties Teaching

assistants for instance have little impact on school results But they create all sorts of other value such as saving the sanity of teachers by helping to look after children in the classroom

Source Teaching and Learning Toolkit Education Endowment Foundation

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 19: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

19

let alone the individual With these potential longer-term benefits it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets they have to worry about where to find the cash for next term not worry about future generations)

Other What Works Centres do take into account the longer-term costs and benefits The Early Intervention Foundation puts lots of attention on future value not just immediate benefits Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future ndash and quantify this via the use of discount rates For wellbeing the proposed discount rate is 15 per cent per annum in line with Department of Health and Treasury Green Book guidelines22

Conclusion

Nestarsquos Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence ndash in whatever form evidence may take ndash as long as it is used by the people that need it Evidence of value is necessary and needed But the whole point is to make measures of value actionable Otherwise we just create new gleaming ivory towers of value metrics that are ignored ndash except by the back-office bean counters

The great advantage of the What Works Centres is how they foreground the user Any teacher governor or parent can for instance grasp the Education Endowment Foundation toolkit If you want more of the research nuance then you can dig down a few clicks for detail It also uses a measure of value that is plausible in the eyes of the public the months gained or lost in the classroom by using certain teaching techniques This is not an abstract monetisation of value but something that matters to schools budget holders and parents

But there is no consistency between the different types of value they use When the Centres were first launched in 2013 there were only six of them Now there are ten (although the one in Scotland is closing) And there are at least four other prospective lsquowhat works wannabeesrsquo who came together at an event in Nesta in 2017 None of these organisations is working towards using a common currency of costs benefits and public value

We need to address this Itrsquos confusing if one social programme is given one measure of value by one organisation and another measure by another organisation The point is to make decisions easier not harder and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes We also need to learn from behavioural research about the best way to communicate numbers and value ndash in a way that lands with stakeholders Using good user-focused design tailored-and-targeted messaging systems that hard-wire evidence and value into daily decision-making will all help Various lsquonudgesrsquo may help citizens and professionals act on value too The Behavioural Insights Team a partnership with Nesta and the Cabinet Office has created nudges to help citizens do things they value but that they have not got around to doing such as enrolling in workplace pensions or signing up for organ donation

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals a project Nesta ran with What Works Wellbeing UCL and the Wellcome Trust Anybody presenting data and research can learn from this research

We should continue to experiment and try out different ways of communicating value evaluate our mistakes and celebrate success ndash like the brave if flawed NICE QALYS There will always be plenty of critics of any attempt to count value but we must go on trying out new approaches and new ways of communicating value

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 20: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

20

2 Measuring value in culture and heritage By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value Most obviously many cultural institutions in the UK such as museums and performing arts organisations at national and local levels receive funding ndash directly or indirectly ndash from the state Cultural objects can also be subject to specific legal protections such as licensing rules on their export or in the case of physical buildings and structures listing and conservation status in the planning system

At the same time there are a number of challenges in measuring the value of culture

bull The frequent absence of a price as a measure of the value to users A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public for example when museums have free entry And even if there is a price as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or peoplersquos valuation of it In the case of cultural objects many may have been acquired without involving financial transaction and if they were sold it may have been a long time ago

bull The value extending beyond direct users The value of culture can extend beyond people who make direct use of it (so called non-use values) For example we may enjoy the view of a historic cathedral even if we do not visit it We may value that a museum exists to protect cultural assets for future generations even if we have never been There is often not a ready measure of these non-use values and of how they vary over the population

bull Uniqueness Some cultural objects are unique so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects For example there is only one Westminster Abbey and one Rosetta Stone

bull The value requiring expertiseexperience to appreciate Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance

bull The value arising from activities which are hard to clearly define There are also certain cultural activities such as historicsocial traditions which do not always correspond to a tangible object or institution so it may not be clear to the public what the thing being valued actually is

bull Social creation of value Some forms of value in culture arise from group participation eg people attending a music concert get a personal value from the experience but this value is affected by the value other attendees obtain making the two hard to disentangle andor aggregate Arguably such forms of social value cannot be assessed using traditional economic approaches based on individualrsquos utilities23

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 21: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

21

More fundamentally there is a widely held view in the cultural sector which is sometimes but not always linked to the issues just described namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure This is despite the fact that implicit financial measures of the sectorrsquos value are made regularly both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects There has also historically been a lack of rigorous evidence on peoplersquos valuation of culture

To explore these issues Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques These studies have focused on cultural institutions and the built environment The main valuation challenges that they have addressed being the absence of a use value (due to free entry) the existence of non-use values and uniqueness The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture while also generating values that can in principle be applied in similar contexts

This research has looked at the value that users and non-users obtain from the following institutions

bull Tate Liverpool and the National History Museum in London24

bull Four regional museums the Great North Museum Newcastle the National Railway Museum York the Ashmolean Museum Oxford and the World Museum Liverpool25 The findings from this are summarised in Box 5 below

bull Four historic cities (York Lincoln Canterbury and Winchester) and their associated cathedrals26

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two as non-usersrsquo valuation may potentially include their valuation of a future visit) The regional museums and historic citiescathedrals studies also investigate the validity of transferring values estimated for one group of sites to others a process known as lsquobenefit transferrsquo

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques ndash hence measure only those benefits that can be captured by traditional economic theory These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants All of the studies use a technique known as contingent valuation where to assess how much they value the sites survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage) The studies use a number of procedures to maximise the robustness of the findings For example the survey design includes information on the cultural asset being valued so that it is in so far as possible an informed decision There are also checks to avoid people giving unrealistic answers for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations Data on the demographics of survey participants is also collected to check the representativeness of the sample and where possible the data weighted to ensure the estimates reflect the appropriate target population

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 22: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

22

Box 5 Valuing usersrsquo and non-usersrsquo attitudes to museums

Table 1 Museum use values and non-use willingness to pay values

The table below shows the average estimates of use value and non-use values as measured by willingness to pay (WTP) among visitors and non-visitors at four regional museums The World Museum (Liverpool) the Ashmolean Museum (Oxford) the Great North Museum (Newcastle) and the National Railway Museum (York) The information was obtained by means of an online survey that is representative of the general population

The willingness to pay estimates relate to the amount that survey respondents are prepared to pay to prevent a hypothetical scenario of museum closure due to funding cuts For museum visitors this is measured by means of willingness to pay an entry fee that would be imposed due to the cuts (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) is framed as a donation

People are counted as visitors in the study if they have visited in the previous three years

The use value as measured by individualsrsquo willingness to pay averaged across all four museum sites is found to be around pound640 for museum visitors and the average non-use value is estimated at pound350

To test the feasibility of applying these values to comparable sites each museumrsquos values are selected in turn and predicted using the average of the estimated values from the remaining museums This is done for all the sites for both users and non-users and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was This and the largest error from all the predictions (the max transfer error) is shown in the table

The broad similarity of values across the sites as evidenced by the comparatively low transfer errors supports the proposition that these values can be

used to estimate the use and non-use values for regional museums that are considered comparable

Population

Visitor

Non-visitor

Museums

Usenon-use value

Use

Non-use

Study site mean WTP (four sites)

pound642

pound348

Mean transfer error

95

168

Max transfer error

182

328

The approach followed is consistent with the best practice set out in the UK Treasuryrsquos Green Book the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken Complying with the Treasuryrsquos guidelines allows the results to be used in business cases in the public sector27

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 23: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

23

Box 6 Valuing usersrsquo and non-usersrsquo attitudes to cathedrals and historic cities

The table below shows the estimated willingness to pay (WTP) values for both users and non-users of historic cities (York Canterbury Lincoln and Winchester) and their associated cathedrals Users including both residents and tourists from within the past three years All cathedral visitorsusers count as city visitorsusers

These estimates are in terms of willingness to make a one-off donation towards measures to reduce the risk of closure and damage buildings in the historic city (including its cathedral) due to a hypothetical scenario of the effects of future climate change Where a historic city value is given the cathedral valuation is obtained using the share of the amount that those surveyed state they are prepared to allocate to the cathedral from their overall city-wide donation otherwise respondents are asked a specific cathedral valuation question

The estimates are obtained by means of an online survey of people who have previously visited the city and cathedral and a survey of the general population that have not visited

The value as estimated by individualsrsquo willingness to pay averaged across the cities is around pound960 and pound615 for historic citiesrsquo use and non-use value respectively For their cathedrals it is just over pound740 and pound370 for use and non-use value

As with the museums study to assess the feasibility of applying the values to other sites each historic city and cathedralsrsquo values are selected in turn and predicted using the average of the estimated values from the remaining cities and cathedrals This is done for all the sites for both users and non-users and the mean transfer error calculated as a measure of accuracy in transferring values This and the largest error from all the predictions (the max transfer error) is shown in the table

As with the regional museums the similarity of values across sites as evidenced by the comparatively low transfer errors supports the

proposition that these values can be used to estimate use and non-use values for historic cities and cathedrals considered comparable

Population

Population

Residentvisitor

Visitor

Non-residentvisitor

Non-visitor

Historic city

Cathedral

Usenon-use value

Usenon-use value

Use

Use

Non-use

Non-use

Study site mean WTP (four sites)

Study site mean WTP (four sites)

pound963

pound742

pound614

pound375

Mean transfer error

Mean transfer error

32

108

125

107

Max transfer error

Max transfer error

66

153

214

193

Table 2 Cathedral and Historic city use and non-use willingness to pay values

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 24: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

24

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public which are based on individualsrsquo self-assessment of their wellness in the moment or in terms of life satisfaction The approach looks at how experiencing certain activities such as visiting a cultural institution affects wellbeing by comparing the stated wellbeing of users and non-users of the institution In principle the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often)

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected

Policy recommendations Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions for example an investment in a cultural facility which expands its capacity such as a museum extension There are also legislative decisions which affect the sector

In principle all such decisions should be informed by economic evidence on the value of these activitiesprojects for society but in practice the extent to which this happens is limited This is partly because of a widespread belief that economic valuation estimates which are at least in principle straightforward to implement necessarily omit essential dimensions of value and for key stakeholders While we share this belief we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture We have long called for the research councils to lead interdisciplinary research in this area to produce more rounded accounts of the value of culture and sharpening our understanding of when economic approaches and other approaches align and collide28 and this gives rise to our first recommendation

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 25: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

25

1 Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making In the UK the forthcoming Centre for Cultural Value from the AHRC Arts Council England and Paul Hamlyn Foundation obviously has an important role to play here although the extent to which it will be resourced to fund the new methodological research needed is unclear Rigorously assessing the value of cultural activity can be costly Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation Even if good evidence is generated it may not be used consistently in policymaking

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity This will involve estimating values on a consistent basis for similar cultural and heritage assetsactivities as done in our regional museums and historic cathedralscities studies to provide benchmark values that can be applied in business cases in the public sector Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact Related to this there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions For example as many cultural amenities in addition to having value for visitors are widely considered to contribute to local wellbeing it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments29

2 Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand peoplersquos valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes Policy initiatives to improve understanding of the publicrsquos values about cultural activity using digital tools represent an exciting new approach to valuation in the sector which should be explored In other areas for example work has already been done in crowdsourcing information to understand perceptions of an arearsquos safety the extent to which peoplersquos surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys30 Computer game environments have also been used to engage people in discussions about local planning decisions31

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 26: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

26

3 Greater use of crowdsourcing and crowdfunding to involve the public in valuation

bull We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making Initiatives could range from

Using information from the public to decide programming at a cultural venue

Making decisions between projects that have already gone through an expert selection process

bull It is however important to acknowledge the challenges involved in ensuring such activity is inclusive as the ability to engage in crowdfunding will not be consistent across the population

A concern relating to the use of valuation techniques in the cultural sector is that in making assessments of value expertise is often needed to make judgements about a cultural activity or objectrsquos technical excellence and originalityimportance The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality32 An assessment of how much this is indeed the case could be built into the trialing of such initiatives

4 Using augmented and virtual reality to provide the public with better information on cultural and heritage facilitiesobjects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality in principle provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys There is also a wider set of issues in the built environment as to how to involve the public in the planning process of which that relating to new cultural and heritage facilities is just one part There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too33

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 27: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

27

3 Realising the value in health By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners including NHS England and The Health Foundation to look at how and why health and care services can work alongside people and communities to create value This section brings together some of what we found

wwwrealisingthevalueorguk

When it comes to health people value more than just clinical outcomes

In health perceptions of value have been dominated by a mix of clinical outcomes system targets competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services Despite being critiqued these approaches continue to have a significant impact on the behaviours of individuals and organisations The current NHS payment system is a highly complex mix of methods prices incentives and penalties

QALYs (quality-adjusted life years) have been viewed with interest by other sectors and for many sectors getting to an equivalent generic measure would be seen as a step forward There are clear strengths to QALYs but they have also faced criticism not least for insufficiently considering the perspective of citizens Alternative ways of understanding value in health such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally The Realising the Value programme set out to explore what lsquovaluersquo means for people and communities and how this can be understood most appropriately by the NHS By doing so the programme sought to orientate the health debate to more actively respond to what people using health services value Our work aligns to the wider movement in health to lsquomeasure what mattersrsquo34

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 28: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

28

People make decisions every day that affect their health and wellbeing

It is people not health and care services that primarily drive the creation of value in health Although medicine and hospitals make a vital contribution the roots of health and wellbeing actually lie in being connected into a thriving community For example in relation to exercise managing stress taking part in social activities or developing skills to successfully look after a health condition And people have family friends communities and peer networks that can work alongside healthcare professionals to support them to live well For example the value of voluntary activity in the UK has been estimated to be pound23 billion35 The economic value of the contribution made by carers in the UK has been estimated at pound132 billion a year36

There is therefore increasing interest in how measures of value in health and care can focus on what is important to people what skills and attributes they have the role of their family friends and communities and given all this what they need to enable them to live as well as possible In Realising the Value we called this taking a lsquoperson- and community-centredrsquo approach

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing independence social connectedness choice and control37 The NHS across the UK has followed For example NHS Englandrsquos Five Year Forward View included a vision to develop new relationships with people and communities in which patientsrsquo own life goals are what count Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7) It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals as well as those that happen in peoplersquos own homes and in the wider community

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 29: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

29

Box 7 Examples of person and community centred approaches in health and care

The Realising the Value programme partnered with five voluntary community and social enterprise

sector organisations that are exemplars in these fields and measure their value in a variety of ways

Approach

Peer support

Health coaching

Asset-based approaches

Self-management education

Group activities

What is it

Approaches through which people with shared experiences characteristics or circumstances provide mutual support to promote health and wellbeing

A form of coaching that aims to help people to set goals and take actions to improve their health or lifestyle

Ways of working that build on and connect existing assets and strengths within people and communities to maintain and sustain health and wellbeing Asset-Based Community Development is an example of an asset-based approach where communities drive the development process and both respond to and create local opportunities

Formalised education or training for people with long-term health conditions which focuses on helping them to develop the knowledge skills and confidence to effectively manage their own health and care

Group activities that can be beneficial to support health and wellbeing such as exercise classes cookery clubs community choirs walking groups and gardening projects

Example of value from Realising the Value local partner site

Positively UK conducted an independent evaluation which found that 80 per cent of participants reported reduced feelings of isolation and were better able to talk to others about living with HIV 95 per cent of participants reported improved wellbeing

A former participant in Being Well Salford said ldquoI am volunteering and attending an art group and I am far more active It is established that I am valued and I have value in life I feel normal again lsquomy normalrsquo You canrsquot put a price on a personrsquos health and wellbeingrdquo

A social return on investment study of Salford Dadz a project run by our partner site Unlimited Potential and featured in the case study below found that pound1 invested yielded approximately pound3 of potential savings to childrenrsquos services and also yielded approximately pound13 of wellbeing value for the fathers involved

Penny Brohn UK have demonstrated a significant change in Patient Activation Measure (PAM) scores among those taking part in their education course with people at the lowest levels of lsquoactivationrsquo before the course showing the greatest levels of improvement

Creative Minds which co-funds creative projects across South West Yorkshire Partnership NHS Foundation Trust uses the Warwick-Edinburgh Mental Wellbeing Scale and has consistently shown improvements in self-reported mental wellbeing across their projects

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 30: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

30

Through Realising the Value we aimed to make more visible the difference that such approaches can make to peoplersquos own health and wellbeing and to the wider system We found that there is growing ndash and increasingly convincing ndash evidence that when people are actively involved in their own health and wellbeing or support others to stay well it can create value for them and the health and care system across a range of dimensions outlined below

Value for people

Realising the Value demonstrated that when people are more actively involved in health their own mental and physical health and wellbeing can be improved including improved clinical outcomes and improvements in peoplersquos confidence to self-manage as well as their wider wellbeing and quality of life We think this value for people strengthens the investment case for person- and community-centred approaches

For example person- and community-centred approaches have been shown to increase peoplersquos self-efficacy and confidence to manage their health and care improve health outcomes and experience and reduce social isolation and loneliness UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life anxiety depression communication and feeling in control of life and able to cope with its challenges38

The case study below demonstrates the value that can be added through people being actively involved in health

Box 8 Case study Alex McCraw Father Community Volunteer Salford Dadz committee member Expert Link Advisory Panel member

I am a 41-year-old father of three I faced many challenges growing up of sexual and mental abuse bullying mental and physical health issues and social isolation I learnt how to survive by cutting myself off from nearly everything and everyone

A couple of years ago I received the best help from somewhere I wasnrsquot expectinghellip A new project was set up to promote the importance of fathersrsquo wellbeing and the impact it has on their kids They put me in direct contact with other fathers who had been through their own struggles some of which were similar to mine Here I found that actually talking about and sharing lived experiences helped me to bring some clarity to my life

Salford Dadz has allowed me to sit down and talk with someone who lsquohas been there and got the t-shirtrsquo

On reflection I understand that when you are at rock bottom it is hard to see any hope But talking to someone who has also been at rock bottom too can help change your perspective as they have a similar lived experience This can benefit you more than talking to a professional where there is not that type of connection or understanding

Moving forward I feel this kind of support should not be offered as a replacement for the system but should be a big part of the wider support system Some of this change in society has started to happen already I hope to be able to help this change happen and that it can continue to lead me and others to a better future

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 31: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

31

Value for the system

In addition to being valuable for people person- and community-centred approaches have the potential to impact how people use health and care services bringing value for the system Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system including reducing demand on formal particularly acute health services as well as achieving savings for local authority care budgets

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing and there are merits (and limits) to this type of analysis A number of organisations have previously sought to quantify the potential global impact For example Reform estimates stronger patient engagement including more self-care improved public health and greater patient contribution to their care could save pound2 billion39 Monitor has estimated that teaching people with long-term conditions to manage their own care could save between pound02 billion and pound04 billion40 And through previous programmes of work Nesta has estimated over pound44 billion a year could be saved if the NHS adopted innovations that involve people their families and communities more directly in the management of long-term health conditions41

The Realising the Value programme undertook economic modelling led by PPL Consulting to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings ndash at the same time as bringing value to people and communities We saw this modelling as a contribution to the debate recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University which collated and analysed systematic reviews of person- and community-centred approaches we found the evidence is relatively stronger for peer support and self-management education approaches In terms of financial benefits our modelling suggests that for people with a subset of particular long-term conditions we could anticipate net savings of around pound2000 per person reached per year There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over pound5 million per year Extrapolating the model further suggests that if implemented well and at scale across England there may be potential for savings of up to pound950 million per year from targeted peer support and self-management education with people with these particular conditions who are expected to see the most benefit

The full impact of investing in person- and community-centred approaches could be significantly higher than this when wider social impacts such as improved employment outcomes and reduced social isolation are taken into account

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 32: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

32

Estimated annual net savings from implementing targeted peer support and self-management education

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (eg evidence that uses control group comparison) The model aims to produce estimates within a clearly defined scope These estimates do not for example extrapolate evidence from condition-specific research on other health conditions or put financial values on outcomes where there is not sufficient evidence to do so While the model uses some of the most robust evidence available the data still has limitations for example studies conducted in one part of the UK may not produce the same results in another part with different demographics or with different levels of existing provision

Source Realising the value (2016)42

Wider social savings are based on offering Peer support to individuals with HIV and self-management education interventions to people with cancer

Savings to the health system are based on providing Peer support to people with mental health issues and coronary heart disease and self management education to people with cardiovascular disease and asthma

Potential wider social savings

mdashpound20800 mdashpound22m mdashpound45b

Estimated savings to the health system

mdashpound2100 mdashpound52m mdashpound950m

Savings per person Savings for one CCG National savings

All five approaches show a range of positive effects for individuals as well as great financial promise The evidence is particularly robust for peer support and self-management education

If these initiatives were offered to people with the four long-term health conditions we have data for they have the potential to offer commission-ers a net saving of mdashpound2100 per person per year

If these interventions were provided at CCG level we estimate that CCGs could save around pound52m per year

This would require theintervention to be targeted carefully at those people who might see the most benefit and implemented well

It is harder to make robust estimates at a national level ndash eg We donrsquot know what has already been implemented

The model suggests that if implemented well and at scale across England there may be potential for savings of up to pound950m per year from targeted per support and self-management education to people with specific conditions who are expected to see the most benefit

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 33: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

33

Box 9 Assumptions in the economic model

Source Realising the value (2016)43

bull Per person savings are calculated using results from a CCG with a population of approximately 250000 people (the average CCG population is 259000)

bull For CCG and national results we have assumed that there would be no overlap between peer support and self-management education benefits especially as they are targeted at different conditions

bull For net financial savings we are only showing benefits for conditions we have evidence for and for conditions that show positive net savings

bull For net wider social benefits we are only showing benefits for conditions we have evidence for and for conditions that show positive net benefits

bull All results use the economic modelrsquos lsquosuggested targeted populationrsquo rather than the total eligible population This is because we recognize that a CCG is unlikely to provide person-and community-centred approaches to all of its residents with a health condition such as cancer or diabetes We assume that CCGs will target the proportion of the population most likely to benefit for example patients with severe health conditions and the ability to commit to an intervention

bull The results assume that person-and community-centred approaches have not yet been implemented in CCGs and there would be capacity in the system to scale up (ie providers would be able to offer the interventions) If a large number of providers are already implementing these approaches there would be lower potential benefit

In todayrsquos financial environment there is significant pressure for approaches to demonstrate return on investment We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system And it could potentially be seen as representing the lsquotip of the icebergrsquo It is plausible to suggest that there may be unaccounted-for benefits because we currently have quantitative financial evidence for a very small number of conditions and approaches If person- and community-centred approaches were offered for more conditions to a wider population and in different forms the savings could be greater Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt

Realising the Value called for further development of the evidence base for person- and community-centred approaches across all three dimensions of value to establish not just lsquowhat worksrsquo but lsquowhat works for who when where and with whomrsquo using rapid experimental methods combined with long-term research

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 34: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

34

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services Person- and community-centred approaches can lead to a wide range of social outcomes from improving employment prospects and school attendance to increasing volunteering They may also contribute to reducing health inequalities for individuals and communities

Although it is plausible to assume that person- and community-centred approaches bring wider social value we found little authoritative work to evidence this impact We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society but our ability to link investments in health and care to these outcomes is limited We feel this is a missing part of the lsquobusiness casersquo for person- and community-centred approaches and that more research and investment in this area would be useful

A broader view of value

Value frameworks ndash and techniques for evaluation and measurement ndash that are capable of capturing all of the valued outcomes and impacts are required These are likely to consist of financial information such as costndashbenefit analyses other forms of quantifiable data stakeholder experience and person-reported outcomes and also qualitative information bound together by narratives that make sense of all the various outcomes as a whole

Realising the Value called for a future articulation of value aligned across health social care and community organisations that takes a broad view in the following ways

Source Realising the value (2016)44

Value broadening the focus

Not only But also

Specific clinical outcomes

What the system values eg cost and value for money indicators

Immediate outcomes of a single service eg success of a treatment

Patient experience ie what contact with services feels like

Individual outcomes for the person

Wider health and wellbeing impacts

What people and their communities value ie the outcomes most important to them

Outcomes over time of all the services and support a person or community may draw upon

Wellbeing eg quality of life people feeling supported in control socially connected and independent

Equity in health and wellbeing with greatest value achieved by targeting people and groups with greater need lower health literacy least access etc

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 35: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

35

Many organisations have already called for a new simplified outcomes framework These include the NHS Confederation the Local Government Association the association of directors of adult social services in England (ADASS) the Academy of Medical Royal Colleges and patient organisations NHS England and others should respond to this cross-sector multi-organisational demand for a change and take action This work will need to engage with the tension between centralisation and standardisation and the diverse perspectives of people and communities This can be addressed by focusing on a small set of outcomes such as independence empowerment social connection the ability to have a family and community life health-related quality of life feeling supported

How people and communities achieve core outcomes like these will differ and some people will want to achieve outcomes beyond those outlined A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance and by building commitment focus and skills for improving it

For areas looking to begin this process we proposed a set of value statements and accompanying pledges that could be adopted immediately by local areas seeking to add value to peoplersquos lives and mobilise the value that people and communities themselves can create for health and wellbeing

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing and the outcomes that are important to them

There is an opportunity for this work to be developed further through NHS Englandrsquos commitment to Personalised Care which has been influenced by the Realising the Value programme The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing community-based support including peer support personal health budgets and importantly a commitment to a different relationship with people based on choice sharing control and understanding what matters to them As part of this there is a commitment to develop metrics and frameworks to capture progress including drawing on citizensrsquo own experiences and perspectives

We value the creation of health and wellbeing

We value people feeling supported in control socially connected and independent

We value peoplersquos contributions (their strengths time effort and skills)

We value sustainable outcomes over time achieved through working together as services and in partnership with people

We value equality and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership

Source Realising the value (2016)45

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 36: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

36

Conclusion

Through the Realising the Value programme we were able to make the case for how and why putting people at the heart of health is valuable for people as well as valuable for the wider community and the system The programme strengthened the case for change and identified evidence-based approaches It also sought to develop tools to support implementation across the NHS and local communities

Based on our learning and insights from the Realising the Value programme we developed ten major recommendations to help make the vision a reality ndash with a focus on both what needs to happen and how the work should be implemented

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people For more information on the Realising the Value programme and our recommendations please see wwwrealisingthevalueorguk

1 Implement person- and community-centred ways of working across the system using the best available tools and evidence

2 Develop a simplified outcomes framework focused on what matters to people

3 Continue to learn by doing alongside further research

4 Make better use of existing levers such as legislation regulation and accountability

5 Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform

6 Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways

7 Develop strong and sustained networks as an integral part of implementation

8 Value the role of people and communities in their health and wellbeing including through co-production volunteering and social movements for health

9 Make greater use of behavioural insights to increase effectiveness and uptake

10 Support a thriving and sustainable voluntary community and social enterprise sector working alongside people families communities and the health and care system

What needs to happen

How people need to work differently

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 37: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

37

Some general lessons By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions Otherwise the risk is that funders and governments rely either on hunches or prejudice or crude monetary measures That is why we favour more transparent and explicit explorations of value

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders or between public agencies and the citizens who benefit from them and pay for them These conversations and negotiations are bound to involve qualities as well as quantities values as well as value As many commentators have pointed out there is an analogy with the electromagnetic spectrum Although radiation can take many forms only a narrow range of frequencies is visible to the naked eye in the form of light Similarly money focuses attention on only some of the features of the world around us and obscures others Since many of the features it obscures matter a great deal to the voting public itrsquos not surprising that the simpler attempts to monetise public value have failed

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public

Some useful principles were developed in one attempt to use public value in UK government which I was involved in46 This work started with a few simple principles

One was that something should only be considered valuable if citizens ndash either individually or collectively ndash are willing to give something up in return for it Sacrifices can be monetary (eg paying taxes or charges) they can involve granting coercive powers to the state (eg in return for security) disclosing private information (eg in return for more personalised services) giving time (eg as a school governor) or other personal resources (eg donating blood)

Some idea of lsquoopportunity costrsquo is essential for public value if it is claimed that citizens would like government or charities to produce something but they are not willing to give anything up in return then it is doubtful that the activity really is seen as valuable For example an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action does not constitute evidence that higher spending will increase public value

Another principle was that the different dimensions of value should not automatically be treated as commensurable Economics has traditionally treated all values in this way anything can be traded off against anything else and turned into a monetary value But many of the currencies governments and societies deal with are not like this Laws ensure that such things as votes body parts and freedoms canrsquot be sold and the public turn out to

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 38: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

38

have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures as well as many environmental measures CBA and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers Useful methods of valuation need to cope with varied types of value with differing degrees of certainty

A more challenging principle is that any measures of value should be comprehensible and plausible to the public Itrsquos not enough for a measure to make sense to specialists If it doesnrsquot help to educate the public about choices and to enrich the democratic process then itrsquos likely at some point to be rendered irrelevant by raw politics

Social and public value effective demand and effective supply

A critical conclusion is that value is never an objective fact Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value Instead these types of value can only be understood as arising from the interplay of supply and demand and through processes of negotiation and argument In consumer markets value is determined by the shifting decisions of individual consumers and the interaction of supply and demand In the public sector itrsquos refracted through political argument and more specifically from the interplay of what I call lsquoeffective demandrsquo and lsquoeffective supplyrsquo (these are essentially what economists call supply and demand ndash I add the adjective effective because in the social and public fields itrsquos important to be clear on the difference between demand and demand that is effective) Effective demand means that someone is willing to pay for a service an outcome or a change in trust That someone may be a public agency a philanthropic foundation or it may be individual citizens Effective supply means that there is a capacity to provide that service outcome or trust a public agency NGO or business

In some fields there are mature links between supply and demand for example public willingness to pay through taxes for policing or primary schools connects to governmentsrsquo ability to supply in familiar ways In other fields the links are missing There may be available supply but insufficient demand ndash because the public or politicians donrsquot see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category) In other cases there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions)

Both sides of the equation may be complex or fragmented In many areas of social policy demand for the results that come from more holistic approaches is split across many different public agencies from welfare to prisons Equally the supply may be equally fragmented depending on the contribution of many different agencies for example providing therapy alcohol treatment skills and housing

In these cases value has to be discovered and a critical role will be played by a lsquosocial market makerrsquo who brings together supply and demand This could be a local council or a national government and in rare cases a foundation or impact investor To do this well they need to help both sides of the market clarify what they want and what they can provide On the demand side public agencies need to work out what they are willing to pay for lower crime in five yearsrsquo time or lower welfare payments On the supply side analysis needs to show what can plausibly be expected from different mixes of actions For this some of the existing Social Impact Assessment or Social Return on Investment methods can be used

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 39: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

39

Then a series of negotiations can take place to commission activities and services with the lsquosocial market makerrsquo working to ensure that no agencies try to free ride on others These processes are likely to work best when they can be disaggregated as well as aggregated eg when commissioning agencies can directly specify what they want from provider agencies

The recent Social Impact Bond on loneliness ndash supported by Nesta and managed by Social Finance ndash is a good example of this Ten years ago there was no effective demand for solutions to loneliness but growing awareness of the direct and indirect costs of isolation has changed this to the extent that the UK now has a minister for loneliness

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works and what is really cost effective Bringing these together has made it possible to use the device of the lsquoSocial Impact Bondrsquo to make public value not only visible but also measurable and monetisable

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset Carbon credits are one version of this and have become a tradable asset Some governments have experimented with currencies in the hands of individuals ndash such as credits for education or learning

We at Nesta have funded various kinds of currency such as Spice Local Exchange Trading Systems and local currencies These aim to make explicit the hidden value in potential local activities and exchanges ndash for example for childcare everyday jobs ndash which are not adequately valued by the monetary economy

Complementary currencies have been with us for decades even centuries They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money) Throughout history different kinds of money have been created sometimes by communities sometimes by companies and sometimes by local governments Their aim has been to connect different kinds of supply ndash of time work or things ndash to otherwise unmet demands or needs

There are now thousands of examples globally Many have thrived when the mainstream economy contracted as happened in Argentina in the early 2000s or Greece in this decade But there are also many examples in countries like Germany where they have become an expression of community spirit

Some of these currencies are like Spice (now named Tempo) explicitly about time They allow us to exchange hours of our time with others At their best they work by formally valuing things that the mainstream economy finds hard to understand That might be the time and skill to care for someone to cook or to fix things

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy that it treats as valueless people who obviously have social contributions to make At a macro scale that failure translates into millions left unemployed or underemployed At a micro scale it means that many communities simultaneously have unmet needs and underused capacities Many of our social ills have their roots in this kind of unnecessary uselessness because people come to internalise the message the systemrsquos implicit message that they have nothing valuable to offer

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 40: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

40

So one route for further research public value is to think through how variants of public value can be turned into currencies or assets The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset Models like Spice turn public value into something more like a currency With a proliferation of new tools for designing and managing currencies ndash some using blockchain ndash this is likely to be a fertile avenue to pursue

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure In other words value will mean different things to different groups some of which have direct power over its definition purchasing or creation and others of whom donrsquot Again this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations

This became apparent to me a few years ago working on a project to define public value in the built environment The field was full of sophisticated tools But we soon realised that there were multiple ways of thinking about value ndash economic social design ndash and multiple stakeholders These diagrams attempted to capture the key points This first diagram provides a basic overview of the kinds of value involved

1 Land value(influenced by local plan and market)

2 Development value(assets rental streams)

Local character

Job creationBike racks

Transport links2 Public value(value for wider stakeholders)

4 Contextual value(contribution and connectivity to wider urban area)

Analysing the structure of value

SeatingCCTV

Servicing

Public car parking

Use(s)

Local propertyfrontage

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 41: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

41

Next we broke these down into more detail both in terms of elements and stakeholders

Localauthority

Workers

Privatespaceviews

Privatespaceviews

Tourists Shoppers Passengers

Non-usersResidents

(other than fromimmediate

vicinity)

Passers by(on foot

bicycle car)

Homelesstravellers

CommunitygroupsPolice

View of local property

View of proposed developments

Uses of proposed developments

Servicing space

Wildlife habitat

CCTV

Public car parking

Kidsrsquo play-space

Water features

Sheltered areas

Seating

Bike racks

Localpropertyowners

Developer

Roadsstreetsrailway lines

Parksgardens

Canalriver walks

Squaresplazasstation

concourses

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 42: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

42

Then we suggested how these might be used to inform negotiations for example over a new urban development

The key point is that this structure will vary in different fields What works for buildings wonrsquot work so well for schools or health Valuation methods are in this sense mapping tools ndash and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional

Stakeholders

DevelopersLocal authorityPublic

Local needs and aspirations

Aims for the specific siteLocal planStated needs

Translated into design briefs

Discussion of options that a developer would invest in

Options

Schemes 1 2 and 3

Option appraisal

Analysis of value in each option using mix of methods

Calculations and deals

S106 PGS developer pays public pay local authority pay BIDS

Methods for negotiation and decision

Multi-critereaanalysis

Multi-critereaanalysis

Hedonic pricingand propertyappraisal

Contingentvaluation andchoice modelling

Furtheroptionconsideration

Evidence base

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 43: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

43

Horizons in time

Another of the many challenges of public value is time how to value something today relative to something in ten or 50 yearsrsquo time This matters particularly for any kind of preventive action

Governments use discount rates to judge when to make investments since a sum of money in five yearsrsquo time is less useful and less certain than an equivalent sum of money now Typically annual discount rates are around 5 per cent These try to reflect both time preferences and also in the more sophisticated versions to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 15 per cent rate to reflect time preferences and 2 per cent to reflect these income effects)

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future A five per cent discount rate values $100 after 30 years at $3585 today and after 50 years at $769 Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now But these discount rates are at odds with how people think and act

Much of our behaviour reflects an implicit lsquohyperbolicrsquo discount rate that starts off very high meaning that we value things in the present much more than the same thing in five or ten yearsrsquo time But in our own lives we generally regret past decisions that applied a high discount rate to future gains and there is some evidence that people apply different lsquomental accountsrsquo to their choices (for example with a different way of accounting for investment in their own education housing pensions or their own children) Most governments also apply quite different discount rates to different phenomena which is why they are willing to invest in future defence education or infrastructures Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change Nicholas Stern author of an influential UK government review argued that the lsquoinherent discountingrsquo of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash) weighting income for the poor more than the rich and for todayrsquos citizens more than future ones since whereas current average global income is around $7000 his forecasts projected average world income in 2100 at around $100000 But in all of these analyses economics has reached its limits It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate) And it cannot explain much of whatrsquos been observed in human behaviour in relation to the future

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 44: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

44

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value In order to be impactful new ideas about public value have to intersect with new approaches to public finance Here there is to put it mildly a problem The dominant methods of public finance have scarcely advanced in recent decades They often miss out key aspects of value discourage investment or cross-cutting action and remain opaque not just to the public but also often to the politicians responsible for making mistakes

Here I summarise what that problem is and how it could be addressed Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term preventive and evidence-based but found few if any collaborators

What happened to public finance

In handling money governments bear some similarities to other organisations but there are also many differences They should want efficiency value for money and transparency But they usually have many more goals more accountability and a different environment of risk

The standard methods used by leading governments around the world were shaped by successive waves of reform These have resulted in models that are essentially input-based top down controls designed to ensure money goes to agreed and approved purposes They are ndash rightly ndash focused on the primary goals of preventing overspending and corruption Against a backdrop of chronic public debt in many countries these traditional aspects of public finance have not lost their relevance Itrsquos just that they address only one part of what governments need to do

A generation ago a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste A wave of reforms associated with the ideas of lsquoNew Public Managementrsquo had great influence from the 1970s-2010s These were substantially influenced by business and encouraged new ways of managing public services and policies privatisation outsourcing co-payments fees and vouchers

These made very little reference to public value They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection or how to structure private finance and partnerships These are still the topics dominating public finance courses and still being promoted by consultants What counts as the leading edge hasnrsquot changed much in two decades

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 45: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

45

Falling behind ndash whatrsquos missing

By contrast much less progress has been made on many other topics that are now of great concern to many governments in cities and nations alike The first two issues follow on from the concerns of the last wave of reforms and are particularly relevant to governments facing serious fiscal pressures

1 Understanding costs and assets there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector including the relationship between average marginal and variable costs despite the pressures of austerity Itrsquos a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis) or how cuts to one service might raise costs for another or what the true pattern of economies of scale or scope might be in different services A related point is the very limited use of balance sheets or analysis of rates of return on assets held by governments (which require national registers of assets preferably as open as possible) Even less use is made of intangible measures in the public sector

2 Making budgets transparent The move towards making public finance more transparent to the public has also made only faltering progress despite some brave attempts (particularly in France) In principle it is becoming easier to tag many items of spending by geography by beneficiary group or topic area Itrsquos not hard to imagine a much more standardised set of data headers for finance making it easier to automate analyses of how funds are being used and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible This would allow a much more granular understanding of public value

The next set of issues point in different directions to how government can be more agile entrepreneurial and effective (rather than just efficient) They include

3 Funding innovation Very little progress has been made on the finance of innovation within the public sector Much more has been done on financing innovation in science and business and for example the World Bank has sophisticated tools for assessing public spending on innovation in the economy But as far as I am aware no government can articulate a coherent approach to funding innovation in its own operations and answering basic questions what optimal levels of spending might be how these might be divided between different activities and successive stages such as incubation and scaling how risk might be managed what target rates of return should be etc Nestarsquos recent survey on lsquoFinance for Innovationrsquo provides a good starting point in describing the methods that are now available all of which require some view of what is valuable

4 Human investment as far as Irsquom aware (and again I would like to be proven wrong) no governments use investment methods to assess spending on education health and training Highly sophisticated tools are used for assessing investment in physical things roads airports and rail Yet spending on people is in many cases investment ndash designed to achieve returns or results many years into the future ndash and social science has become steadily more competent at analysing the links between actions now and long-term effects Instead spending on fields like health is organised year to year with no accounting distinction between spending on a vaccination programme versus emergency care

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 46: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

46

5 Finance tied to outcomes despite several decades of experiment there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals transparently assessing when these are met or not and establishing budgetary implications) The spread of new tools for funding outcomes is providing a useful prompt for progress including social impact and development impact bonds payment for outcomes and the growing field interested in analysing the long-term returns achieved by different interventions

6 Evidence Although some progress has been made in recent years in orchestrating evidence in public services and making more use of data these methods have not permeated the day to day work of many officials responsible for public finance Itrsquos interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals Theyrsquore not particularly hostile and some are enthusiastic But they donrsquot see this as core to their jobs Yet in the future finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved helped by the key budgetary and financial committees in other words acting as guardians of public value

7 Public engagement We now have the benefit of several decades of experience of participatory budgeting (PB) allowing the public to directly assert what they believe to be valuable Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions At one point the UK alone had well over a hundred PB pilots underway In Paris the Mayor opened up 100m Euro each year to PB and other big cities have experimented too The challenge faced by all of these experiments is how to make budgets comprehensible how to show the current balance of spending how to help deliberation on the impact of shifts of budget from one category to another and how to get the right mix of numbers visualisations and prose Exactly the same challenge of course applies to helping committees of politicians The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation Yet together these offer the prospect of making budgets significantly more comprehensible

8 Funding as a platform Governments have for many years discussed how to make funding flows to individuals more flexible and useable Some like Denmark and Singapore created personal accounts India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance

The failure to tie finance into evolving ideas about public value and the related debates on innovation and evidence risks becoming a major barrier to progress There is a need for better theory and better tools that can be applied and shared by the leading practitioners and put to use in governments at every level whether national regional or city and district This is the necessary complement to further work on public value

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 47: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

47

Sociology may offer additional insights If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments Very strong commitments eliminate the difference between the present and the future even though there is still the same uncertainty as in commercial markets Parents may commit all to their childrenrsquos future aristocratic landlords commit to passing on richer estate than they inherit and for a committed NGO or social movement it is simply inappropriate to devalue future rewards ndash the cause is everything Similarly where fundamental rights are involved it is inappropriate to devalue the future Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020 if they didnrsquot they would build in a profound bias in favour of the old and against the young The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors It is in fact quite culturally specific

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare both of which involve profound moral commitments They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies They remind us that value is not abstract but concrete ndash it exists in real places at particular moments in time and in a social context There is no such thing as intrinsic value It is a consequence not a cause

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun and we value in many ways whether through how we allocate our time our money use political influence or make commitments

As such value has only a loose relation to price Many things are very valuable but not priced ndash like air or a relationship to God Price and the tools of economics become relevant in particular conditions of scarcity and organisation So for example we might value seeing the Aurora Borealis ndash but itrsquos rare and not organised and therefore not easy to value in economic terms

Yet for all that formal valuation can help us to make choices and can make key issues more visible Our case studies are deliberately varied rather than reflecting a single approach But they point to some common conclusions

First that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done They are useful if they bring choices and trade-offs to the surface useless if they disguise them

Second more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together In a democracy that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences even if sometimes they also have to challenge the publicrsquos beliefs

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 48: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

Public Value How can it be measured managed and grown

48

Third value is an aspect of the relationship between states and citizens rather than an objective fact It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation

In any real-world situation trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation alternatives are available in use and steadily becoming more compelling47

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 49: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

49

Public Value How can it be measured managed and grown

Annex Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area Health and social care

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p18)

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis

bull Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes

bull Number of QALYs obtained in evaluation interventions costing the NHS less than pound20000 per QALY gained are cost effective and those costing up to pound30000 per QALY gained might with certain conditions satisfied be considered cost effective

They are also interested in value attributed by publics and social value judgements made by experts ndash judgements that take account of societyrsquos expectations preferences culture and ethical principles

bull Publicsrsquo perspective currently being used in the New approach to cost-effectiveness required for end-of-life drugs

bull Social value judgements they developed a guide on how to use social value judgements

Critiques to NICErsquos approach to value for money

bull From NICE Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements)

bull They do not take into consideration potential costs falling on other public sector budgets

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 50: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

50

Public Value How can it be measured managed and grown

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area Educational attainment

Strong approach

Value for money critical to schoolsrsquo ability to secure improvements in pupil outcomes Cost and cost effectiveness is important for schools (Cost Guidance)

Cost-effectiveness strong element in their approach to policy interventions

Concept

bull It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness

bull Practitioners need to know which approaches will secure the biggest improvements in their pupilsrsquo learning for the lowest cost

bull It is part of their evaluation toolkit cost rating five-point scale to show cost-effectiveness of different approaches

Actions

They developed a guide on cost evaluation for schools and practitioner

It is part of their strategy for the next five years Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan)

Early Intervention Foundation (established 2013)

Policy area Early intervention

Soft approach

Value for money cost of the intervention

Concept of cost

bull Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards)

bull They assess cost in terms of ie mapping local spending

bull Cost is not part of evidence We know that national and local decision-makers need to take other factors into account such as cost and local context (EIF strategy)

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 51: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

51

Public Value How can it be measured managed and grown

What Works Centre for Crime Reduction (established 2013)

Policy area Crime

Strong approach

Value for money it can be achieved using their guidance on how to use evidence It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction)

Cost benefit is one of the dimensions included in their EMMIE model lsquowhat direct or indirect costs are associated with the intervention and is there evidence of cost benefitsrsquo

Including cost benefits evidence = high quality of evidence

Cost effectiveness is a recommended measure for all policy interventions (Reporting guidance)

What Works Centre for Local Economic Growth (established 2013)

Policy area Local economic growth

Strong approach

Value for money use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews)

Cost effectiveness is a key element to understand the causal effects of policy interventions

They only consider impact evaluations

What Works Centre for Wellbeing (established 2014)

Policy area Wellbeing

Mixed approach

Value for money better allocation of budgets and resources but money cannot be the sole guide to decision-making

Other values to take into consideration respectful of rights culture and dignity protection to vulnerable groups social and community cohesion long-term sustainability (Wellbeing and cost effectiveness)

Cost effectiveness they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions

They developed a guide on how to do cost effectiveness in wellbeing

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 52: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

52

Public Value How can it be measured managed and grown

Centre for Ageing Better (established 2015)

Policy area Ageing

Soft approach

Cost benefit they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach Work to be delivered over 2018-10 (Strategy)

Cost effectiveness does not appear in their approach to evidence or their approach to evaluation

Wales Centre for Public Policy (re-launched June 2017)

Policy area Welsh public services

Approach not defined

Cost analysis used in one of their projects

What Works Centre for Childrenrsquos Social Care (established 2017)

Policy area Childrenrsquos Social Care

Strong approach

Value for money costs and benefits of interventions It is an expectation from the public

Cost benefit economic implications of using an intervention (Outcomes framework)

One of their six principles of their outcomes framework

Key for research and service delivery

What Works Scotland (established 2014)

Policy area Scottish local services

Alternative approach

Value for money no clear definition

Cost effectiveness there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles

ie It is clear from this evaluation that participatory processes require time and resources and do not necessarily improve efficiency and reduce costs at least not in the short term (Placed-based evaluation)

ie Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions (Outcomes based approach)

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 53: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

53

Public Value How can it be measured managed and grown

Endnotes1 Fisher W P (2010) lsquoMeaningfulness Measurement Value

Seeking and the Corporate Objective Function An Introduction to New Possibilitiesrsquo Berkeley CA University of California httpsssrncomabstract=1713467

2 Fujiwara D and Campbell D (2011) lsquoValuation Techniques for Cost Benefit Analysis Stated Preference Revealed Preference and Subjective Well-Being Approachesrsquo London HM Treasury httpsassetspublishingservicegovukgovernmentuploadssystemuploadsattachment_datafile209107greenbook_valuationtechniquespdf

3 For example one study of an urban regeneration project was found to have achieved an improvement equivalent to pound19000 per person of working age yet lsquowillingness to payrsquo studies came up with a figure of only pound230 per yearndash Dolan P and Metcalfe R (2008) lsquoComparing willingness-to-pay and subjective well-being in the context of non-market goodsrsquo LSE CEP Discussion Paper No 0890 London School of Economics httpeprintslseacuk285041dp0890pdf For a further study comparing willingness to pay estimates see also Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC httpsahrcukriorgdocumentsproject-reports-and-reviewsmeasuringeconomicvalue

4 Kornberger M Justesen L Madsen A K and Mouritsen J (2015) lsquoMaking Things Valuablersquo Oxford Oxford University Press

5 Flyvbjerg B Bruzelius N and Rothengatter W (2003) lsquoMegaprojects and risk An anatomy of ambitionrsquondashCambridge Cambridge University Press

6 Gintis H Bowles S Boyd Rndashand Fehr E (Eds) (2005) lsquoEconomic learning and social evolution Moral sentiments and material interests The foundations of cooperation in economic lifersquo Cambridge MA MIT Press httpswwwumassedupreferengintismoral20sentimentspdf

7 A study in Michigan for example found a significant link (a correlation of 026) between perceptions of procedural justice in government services as distinct from outcomes and trust in politicians Miller A and Listhaug O (1999) lsquoPolitical Performance and Institutional Trustrsquo In Norris P (Eds) lsquoCritical citizens Global support for democratic governmentrsquo Oxford Oxford University PressndashThere are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government Demery L Ferroni M and Grootaert C (Eds) (1993) lsquoThe Social Effects of Policy Reform A World Bank Studyrsquo Washington DC The World Bank

8 Further experimental evidence from the OECDrsquos Trustlab shows lsquoan increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per centrsquondashMurtin F et al (2018) lsquoTrust and its determinants Evidence from the Trustlab experimentrsquo OECD Statistics Working

Papers No 201802ndashParis OECD Publishing httpsdoiorg101787869ef2ec-en

9 The Centres provide a variety of products to communicate their evidence The Crime Reduction Toolkit the EIF Guidebook EEFrsquos Teaching and Learning Toolkit and the Early Years Toolkit the Growth Toolkits and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre All of these provide information on impact cost and strength of evidence The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts

10 Casey D A Antimisiaris Dndashand OrsquoBrien J (2010) Drugs for Alzheimerrsquos disease are they effective lsquoP amp T a peer-reviewed journal for formulary managementrsquo 35(4) 208-11httpswwwncbinlmnihgovpmcarticlesPMC2873716

11 This was subsequently toned down in later guidance although the drug is only offered to those with moderate or severe Alzheimerrsquos httpswwwniceorgukguidanceta217chapter1-Guidance

12 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

13 Neumann P J and Weinstein M C (2010) lsquoLegislating against use of cost-effectiveness informationrsquo New England Journal of Medicine 363 p 1496

14 This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention 3000 members of the public help score the different dimensions of quality of life from EQ-5D See Ogden (2017) for description

15 This doesnrsquot mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research ndash they do allow people to convert evidence from other wellbeing measures into equivalent lsquounitsrsquo of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways

16 Source Puttick P and Ludlow J (2012) lsquoStandards of evidence for impact investingrsquo Nesta

17 lsquoEvidence vs Democracy How lsquomini-publicsrsquo can traverse the gap between citizens experts and evidencersquo Alliance for Useful EvidenceANSA 2019

18 Coutts P and Brotchie J (2017) lsquoThe Scottish Approach to Evidencersquo Nestandashand Carnegie Trust

19 Kirk P and Shutte AM (2004) Community leadership development lsquoCommunity Development Journalrsquo Vol 39 No 3 pp 234-251

20 Halsam D (2016) lsquoThe NICE Citizens Council and the role of social value judgementsrsquo Presentation available at httpwwwidsihealthorgwp-contentuploads201605Citizens-Council-and-Boardpdf

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 54: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

54

Public Value How can it be measured managed and grown

21 Adultchild ratios in pre-school provision tend to be higher than in school classes - so harder to justify Family interventions have similarly high costs The average cost per child of a Sure Start Local Programme was pound1300 in 2009-2010

22 Smith D and Gravelle H (2000) lsquoThe Practice of Discounting Economic Evaluation of Health Care Interventionsrsquo CHE Technical Paper Series 19 The University of York

23 Throsby D (2001) Economics and Culture Cambridge University Press

24 Bakhshi H Fujiwara D Lawton R Mourato S and Dolan P (2015) lsquoMeasuring Economic Value in Cultural Institutionrsquo Swindon AHRC

25 Fujiwara D Bakhshi H Mourato S Hotopp U Lawton R Lagarde A andndashDavies J (2018) lsquo The Economic Value of Culture A Benefit Transfer Studyrsquo London DCMS

26 Lawton R Fujiwara D Mourato S Bakhshi H Lagarde A andndashDavies J (2018) lsquoThe Economic Value of Heritage A Benefit Transfer Studyrsquo London Nesta

27 HM Treasury (2018) lsquoThe Green Book Central Government Guidance On Appraisal And Evaluationrsquo London HM Treasury

28 Bakhshi H (2012) lsquoMeasuring Cultural Valuersquo London Nesta

29 Davies J (2017) lsquoOptimal designs What are some of the lessons of grand landmark buildings projectsrsquo London Nesta

30 Naik N Philipoom J Raskar R and Hidalgo C (2014) lsquoStreetscore-Predicting the Perceived Safety of One Million Streetscapesrsquo CVPR Workshop on Web-scale Vision and Social Media MacKerron G (2012) lsquoHappiness and environmental qualityrsquo PhD thesis LSE Brynjolfsson E Eggers F and Gannamaneni A (2018) lsquoUsing massive online choice experiments to measure changes in well-beingrsquo NBER Working Paper 24514 Cambridge MA NBER

31 Mather L andndashRobinson P (2016) Civic Crafting in Urban Planning Public Consultation Exploring Minecraftrsquos Potential lsquoInternational Journal of E-Planning Researchrsquo

32 Baeck P Bone J and Mitchell S (2017) lsquoMatching the crowd Combining crowdfunding and institutional funding to get great ideas off the groundrsquo London Nesta

33 HM Government (2018) lsquoIndustrial Strategy - Creative Industries Sector Dealrsquo London HM Government

34 For example the work of the Institute for Healthcare Improvement and the Realistic Medicine movement in Scotland

35 Payne C (2017) lsquoChanges in the value and division of unpaid volunteering in the UK 2000 to 2015rsquo Newport ONS

36 Yeandle S and Buckner L (2015) Valuing carers 2015- the rising value of carersrsquo supportrsquo London Carers UK

37 For example captured through the Care Act 2014 focus on wellbeing prevention personalisation and co-production

38 For further examples see httpswwwnestaorgukreportimpact-and-cost-economic-modelling-tool-for-commissioners

39 Corrie C and Finch A (2015) lsquoExpert Patientsrsquo London Reform

40 Monitor (2015) lsquoClosing the NHS funding gap how to get better value health care for patientsrsquo London Monitor

41 Nesta (2013) lsquoThe business case for People Powered Healthrsquo London Nesta Innovation Unit PPL

42 Realising the value (2016) lsquoTen key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

43 Realising the value (2016) lsquoRealising the value Ten key actions to put people and communities at the heart of health and wellbeingrsquo London Nesta

44 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

45 Realising the value (2016) lsquoNew approaches to value in health and carersquo London Nesta

46 Published as lsquoCreating Public Valuersquo by the Cabinet Office Strategy Unit in 2002

47 [i] This piece draws on several main sources including lsquoCreating Public Valuersquo by Geoff Mulgan Gavin Kelly and Stephen Muers published by the UK Cabinet Office in 2002 a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan Gareth Potts Matthew Carmona Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website) The website wwwsocialinnovatorinfo provides details of many measurement methods alongside several hundred methods in use around the world for every aspect of social innovation A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields [ii] lsquoThe impact of subjective wellbeing on local authority interventionsrsquo Dolan and Metcalfe February 2008 [iii] The following books provide a good overview Barrow C J (2000) lsquoSocial Impact Assessment An Introductionrsquo London Arnold Becker H and Vanclay F (2003) lsquoThe international handbook of SIArsquo Cheltenham E Elgar Becker H A (1997) lsquoSocial impact assessment method and experience in Europe North America and the developing worldrsquo London UCL Press Scholten Nicholls Olsen Galimidi (2006) lsquoSROI A Guide to Social Return on Investmentrsquo Lenthe Publishers A good overview of the rigorous end of assessment and evaluation is lsquoIdentifying the environmental causes of disease how should we decide what to believe and when to take actionrsquo (Academy of Medical Science 2008) A recent study of tools used by foundations is lsquoLearning for Social Impactrsquo from McKinsey 2010 one of a number of recent moves to advocate separate tools for NGOs and foundations an approach which I question For philosophical perspectives John Deweyrsquos lsquoA theory of valuationrsquo and Luc Boltanski and Laurent Thevenotrsquos lsquoOn Justification economies of worthrsquo are highly recommended

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS

Page 55: Public Value: How can it be measured, managed and grown? · Public Value: How can it be measured, managed and grown? This paper documents work in progress but we hope its concepts,

58 Victoria Embankment London EC4Y 0DS

+44 (0)20 7438 2500 informationnestaorguk nesta_uk wwwfacebookcomnestaukwwwnestaorguk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091 Registered as a charity in Scotland number SCO42833 Registered office 58 Victoria Embankment London EC4Y 0DS