Does PPI make a Difference? Measuring the Impact of Involvement Professor Jonathan Tritter All Wales Joint PPI Conference Lampeter University 27 January

Does PPI make a Difference? Measuring the Impact of

Involvement

Professor Jonathan Tritter

All Wales Joint PPI ConferenceLampeter University

27 January 2010

2Warwick Business School

What are we going to talk about?

Defining Public and Patient Involvement

Conceptualising Public and Patient Involvement

What is the impact of Public and Patient Involvement

Measuring Public and Patient Involvement

The way forward


Ways in which patients or clients can draw on their experience and members of the public can apply their priorities to the evaluation, development, organisation and delivery of health and care services

Patients/clients as individuals

Carers on behalf of others

Members of communities, localities and the public

What is Involvement


Involvement in decisions about treatment and care

Involvement in service developmentPlanning, prioritising and commissioning services

Involvement in the evaluation of service provisionRegulation and public accountability not patient satisfaction

Involvement in teaching

Involvement in researchAt all stages of the research cycle

Conceptualising Involvement


Individual and Collective InvolvementWriting to an Assembly Member about the availability of a particular drug or treatmentJoining a support group or community organisation

Proactive and Reactive InvolvementVolunteering at a local hospiceTaking part in a local consultation on service reconfiguration

One-off or Continuous InvolvementAttending an open-meetingBuilding relationships within a system of involvement

Forms of Involvement


Direct InvolvementPeople play a role in making decisions

Indirect InvolvementPeople are sources of information which influences decisions

Vast majority of involvement is IndirectMethods to solely to collect views and experiencesDecision and justification reserved and not transparent

Forms of Involvement


Patients and Service UsersIndividual treatment or care servicesInformed consent

CarersCo-productionEfficacy of treatment or care servicesPromote compliance

Members of the Public: Potential service users

Prioritization of health and care needsResponding to the broader public health agenda

Competing justifications for involvement


LegitimacyOf decisionLessens conflict and resistance to change

RelevanceDifferent kinds of questionsFocus on process and experience not just outcomes

ImpactEfficiency and effectiveness of decisionMore acceptable process for patients, members of the public and professionalsSupporting co-production of wellbeing and compliance with professional advice

Why involve members of the public?


Involvement is about creating accountabilityPromoting the transparency of decision-making

Distinction between democratic accountability and other forms of accountability

Democratic accountability not necessarily the highest form of accountability

Both challenged by apathy

Both challenged by lack of representativeness

Why involve members of the public?


Social goals (Beierle 1999)

Increasing the expertise of participantsIncorporating public values into decision makingIncreasing trust in a decisionImproving substantive quality of decision because of ‘local’ knowledge

Expertise of organisations, professionals and managersImproves decision- making processesGreater care taken with procedures and underlying arguments in cases that generate greater public participation

Indirect Effects of Public Involvement


IndicatorsWhat do you measure

Outcomes of involvementHow does involvement effect the organisation, staff, users and the public?

Impact of involvementWhat is changed because of involvement?Are these changes improvements?

How to Measure Involvement


The purpose of involvementThe purpose of specific involvement activitiesThe purpose of an involvement system

The process of involvementWho participatesThe intensity of involvement activitiesThe experience of involvement

Dimensions of Involvement


The outcomes of involvementIdentification of issuesSuggested improvements

The impact of involvementThe implementation of issues emerging from involvement activitiesThe evaluation of ‘changes’



Indicators need to be:

RealisticMeasurableSpecific

“Each of these constructs lends itself directly to an evaluatory question:

‘did they listen?’, ‘did I get what I wanted?’ or ‘did the service change?’” (Crawford 2003: 15)



The Impact of Involvement

Change that is made because of involvement activities

Change that is ascribed to involvement



What about the intrinsic benefits of involvement?

“such as improvements to self esteem and changes in attitude”

(Crawford 2003: 9)

Should we try to measure these changes?

Do these changes count?



“The idea of citizen participation is a little like eating spinach: no one is against it in principle because it is good for you.” (Arnstein, 1969: 216)

So why do we need to measure PPI?

Measuring Involvement


Barriers to measuring involvement are different from the barriers to doinginvolvement

An ethical issue; involvement is an inherent good so there is no need to measure it

Legal requirement; there is no need to evaluate something that must be done anyway

Participation is a right; there is no need to evaluate a right

Barriers to Evaluating Involvement


Perceived costs and opportunity costs; money spent on evaluation means less can be spent to support involvement

Power differentials have to be considered in the evaluation and make evaluations more difficult and potentially risky

A challenge to existing hierarchies

Cultures of some organisations may obstruct evaluation

Ascription of outcome to involvement may be problematic

Barriers to Evaluating Involvement


Part of the work of the Four Nations PPI Group

Northern IrelandFunded by the NI Department of Health, Social Services and Public Safety and the participating TrustsWorking with four Trusts across Northern Ireland

We conducted 51 interviews with 66 respondentsReviewed Trust documentsObserved Trust events

Initiated by NHS Centre for InvolvementWorking with: Dr Sara Wilford, Dr Jayne Taylor and Connie Lord

Parallel project in England reporting in early 2010

Evaluating the Impact of Personal and Public Involvement: Piloting a PPI

Evaluation Framework


Evaluating the Impact of Personal and Public Involvement

Evaluation of PPI using two tools

PPI Self-AssessmentCompleted by PPI Operational LeadReflectiveQualitative

PPI Performance Management FrameworkOrganisational levelEvidence-ledFixed format


PPI Self-Assessment

Exploring two examples of PPISuccessful Involvement – that made a differenceUnsuccessful Involvement – did not make a difference

Characteristics of two examplesAimsNature of activityParticipants


PPI Self-Assessment

ImpactStaff, service users and members of the publicEvaluation of impact

Reflecting and learning from the processHow would it be done differently if repeated?


Different Dimensions Meeting key performance indicators supported by evidence

Infrastructure

Organisational processes

Impact on organisational decisions

Impact on stakeholders

PPI Performance Management Framework


Involvement is a process not an activityLearn through doing

Involvement is predicated on collective not individual benefit

The centrality of the co-production of public value

Changing the ‘culture’ of organisational decision-making

Relationships between the users and providers of public servicesCreating a different form of partnership

The unvoiced and uninvolvedNot a response to democratic deficit

Reflections


We need systematic approach to evaluating PPITo document the benefits of involvementTo document the costs of involvement

To be able to make a business case for PPI

We need to move to Evidence Based Involvement

Reflections


By postInstitute of Governance and Public ManagementWarwick Business SchoolUniversity of WarwickCoventry CV4 7AL

By [email protected]

Get in touch

Documents

Does PPI make a Difference? Measuring the Impact of Involvement Professor Jonathan Tritter All Wales Joint PPI Conference Lampeter University 27 January