User Involvement in Mental Health Research

Diana Rose and Jenny WalkeService User Research Enterprise (SURE)

Institute of PsychiatryKing’s College London

Purpose

To demonstrate how service user-focused research can add something to the methods and findings of conventional research

To show why service user researchers are best-placed to do this

Involving service users at all stages in the research process

Overview A little bit about one of us (DR) Some introductory slides Service user research

User-focussed monitoring Systematic reviews Outcome measures

Criticisms and answers The NIHR continuum

Having a double identity (DR) Service user all my adult life

1972-1986 academic career

1986 medically retired – ‘living in the community’

1996 two identities come together to do user-led research

Colney Hatch Lunatic Asylum (est 1854)

Friern Hospital corridor 1976 – I walked the line

Friern Hospital patient (1984)

Camden Mental Health Consortium Resettling Friern Patients

Camden – central and north London Friern – patients to be resettled to this

area Camden Mental Health Consortium –

one of first local user groups Formed to make sure the resettlement

was done well Not user-led at this point – that came

later

Service user research - history 1988 – CMHC and GPMH project on new

inpatient provisions in District General Hospital

1996 – two user-led projects based in NGOs Strategies for Living (Mental Health Foundation) User-Focused Monitoring (Sainsbury Centre for

Mental Health) Political: workers members of the user

movement and took research questions from the movement

To make a contribution – our ‘evidence’ for the movement

User research moves on (or backwards) Early 2000s two groups in universities

SURESearch (Birmingham) Service User Research Enterprise (SURE) Service user researchers have ‘insider

knowledge’

SURE has ‘collaborative’ management structure – one conventional university researcher and one user researcher

SURESearch more complex but ‘head’ is conventional university researcher

Activity

Now test your knowledge - correct answers revealed on final slide.

1. What was the original name of Friern Hospital?

a)Bedlam

b)Colney Priory

c)Colney Hatch Lunatic Asylum

2. How many patients did Friern accommodate by the 1950s?

a)500

b)2000

c)5000

3. What does CMHC stand for?

a)Community Mental Health Clinic

b)Centralised Mental Hospital Care

c)Camden Mental Health Consortium

4. When were two reports from service user-led projects published?

a)2000

b)2002

c)2005

Further Reading

Taylor B: The Last Asylum. London: Penguin; 2014.

Survivors’ History Group, Mental health and survivors' movements and context: http://studymore.org.uk/mpu.htm

User-Focussed Monitoring

Method of Peer-Review of Mental Health Services Started in Central London in 1996

Community services Hospital services

Expanded across UK and across service types

Now also used in Nordic countries Norway “User ask user”

UFM is User-Led Co-ordinator who is a user but also with

research skills

Visits local area to identify service users interested in project

Get together over lunch to make final decision about whether they want to be involved

Some UFM projects questionnaire-based

Group devises questionnaire on the basis of their experience of the service to be assessed

Takes many meetings as co-ordinator synthesises ideas and then brings back to group for amendment

Training

Some involved in UFM projects have never done an interview

Some basic training

But mostly role-play: opportunity to experience answering as well as asking the questions

More qualitative projects

Focus groups have been used

Topic guide devised in the same way as for questionnaires but shorter as want to give group scope to expand on their views

Some training for this too

Collecting the information Interviewers go to many different

venues: CMHTs, hospitals, participants’ homes, charities

Focus groups not usually held in NHS venues – more neutral

De-briefing – co-ordinator speak with interviewer immediately after each interview to check no problems

Analysing the data

For questionnaires there is some statistical analysis

Problematic from involvement perspective as few service users have these skills

Easier for members of UFM team to be involved in analysing qualitative data Can bring their experience to bear

What happens to the report? UFM projects are locally commissioned

e.g. Trusts, local authorities, charities Report goes to commissioners Up to them what they do with it

Some take very seriously and results in measurable changes

Some take not so seriously! Sometimes have feedback days for staff

Activity

5. When did peer review of mental health services start in Central London?

a) 1983

b) 1990

c) 1996

6. Why aren’t focus groups typically held in NHS venues?

a) Lack of space

b) Lack of neutrality

c) Lack of insurance

7. At what level are UFM projects commissioned?

a) Local

b) National

c) International

Further Reading

Rose D, Fleischmann P, Schofield P: Perceptions of User Involvement: a User-Led Study. International Journal of Social Psychiatry 2010, 56(4):389-401.

New method – patient-centred systematic reviews

Example - ECT Consumers’ perspectives on ECT Two main researchers had experienced ECT – “insider

knowledge”

Adapted method of systematic review to make it user-focused Included peer-reviewed literature as normal in systematic

reviews Included the ‘grey’ literature Included qualitative data – ‘testimonies’

Assembled 26 clinical papers and 9 authored by service user groups

Testimonies sourced from a video archive and the internet

Main themes

Perceived benefit

Retrograde memory loss

Information

Consent and perceived coercion

Perceived benefit Conventional research showed much higher

levels of satisfaction with ECT than user research – no overlap in the estimates of benefit between the two groups

Critique of method in grey literature

We answered with critique of method in clinical papers

ECT continued BOTH conventional and grey literature showed

high levels of memory loss though conventional did not discuss this or even said it was not important

BOTH showed people did not feel informed but again this absent from discussion in clinical papers

Perceived coercion – testimonies showed some people felt coerced into signing form

Same results but different interpretations – not like the finding on perceived benefit where there were different results

The Dispute

Paper published in a high-profile medical journal

Royal College of Psychiatrists issued press release disputing what we had said about memory loss

Mendacious – they mis-quoted themselves

Policy NICE new guidelines on ECT as our

research work being done Consumer review influenced Especially around information and

consent including about risk of memory loss

User research can have an effect on national policy

Activity

8. Which two of the following are omitted from orthodox systematic reviews?

a)Grey literature

b)Randomised controlled trials (RCTs)

c)Peer-reviewed literature

d)Qualitative testimonies

9. Issues of consent and memory loss are associated which of these treatments?

a)Cognitive Behavioural Therapy (CBT)

b)Occupational therapy

c)Mindfulness

d)Electro-convulsive therapy (ECT)

Further Reading

Rose D, Fleischmann P, Wykes T: Consumers' views of electroconvulsive therapy: A qualitative analysis. Journal of Mental Health 2004, 13(3):285-293.

Patient Generated Patient Reported Outcome

Measures (PG-PROMs)

The claim of neutrality in Randomised Controlled Trials

(RCTs)

RCTs considered the ‘gold standard’ in medicine

Neutrality depends on blinding (not knowing certain details)

But is everything in an RCT neutral?

Outcome measures devised by clinicians and academics

May not be the outcomes that matter to service users

Try to develop measures that are valued by service users (and others) in mental health

Patient Reported Outcome Measures (PROMs)

Much talk of PROMs

But only filled out by patients – no say in which questions are asked

Patient-Generated PROMs (PG-PROMs) Medication side-effects Experiences of inpatient care

Method Participatory research

Attempts to reduce the power relations between researcher and researched

In user-focused research, researchers have the same or similar experiences as the participants

All are mental health service users

A new development even within participatory research

Potential influences on treatment and policy

Procedure Focus Groups which meet twice Recruited because they have experience of what the

measure is attempting to tap Facilitators/researchers have experience of the

treatment or service that is being evaluated On basis of focus group discussions, researchers draw

up draft measure Taken to Expert Panels for amendment and refinement

and that the language is their own language Feasibility study to make sure it is easy to complete –

refinements all the way Psychometric testing About 200 participants involved in all

Example – in-patient care Much anecdotal evidence and evidence

in the ‘grey’ literature that profoundly disliked

My experience too Wanted to do something more rigorous Finally became the main outcome

measure in an RCT evaluating the introduction of psychological therapies on acute wards

Collaborative but our part is user-led

User Measure: VOICE Participants

People who had been in-patients in the local Trust within the previous two years

One group specifically made of participants who had been detained

Focus groups 6-8 people: 4 groups

One facilitator with experience of in-patient care and other also a service user

Meet twice to make sure we have accurately captured their views

Thematic analysis using Nvivo software after both 1st and 2nd wave

Drafting the measure

Done by the researchers on the basis of the qualitative analysis

Quantitative and qualitative questions

Researchers also use own experience

Expert Panels One drawn from focus group members

and one independent

Also been inpatients in previous two years

Tasked with amending and refining the measure and making sure that language and layout are appropriate

Usually quite a few changes made at this stage

Feasibility study

Final stage of constructing measure ~50 people complete the measure

and we find out which parts are easy to complete and which not

Iterative process In this project the participants were

actually in hospital

Psychometrics Mixed methods research

We do this because it is appropriate but also to show that user-led research can be rigorous

One mainstream researcher said users could never produce measures because they would never understand factor analysis

Psychometrics Continued Test-retest reliability: the same people fill in the

measure twice with an interval of a week in between Are the scores the same the 2nd time? This would

mean the measure is stable. VOICE very stable People in this exercise mostly a diagnosis of

psychosis which might compromise stability. Good test-retest reliability because measure

developed by service users?

Psychometrics cont Criterion validity: assess measure against an

existing one Expect some differences because of means

of development: close relationship but there were differences

Acceptability: how does it feel to fill it in Enjoyable? Distressing? Right length?

Activity

10. Who compiles most existing outcome measures?

a) Patients’ families and carers

b) Clinicians and academics

c) Hospital administrators

11. What is ‘blinding’?

a) Conducting research without obtaining consent

b) Assigning participants to different groups

c) Temporarily withholding information to reduce bias

12. What name is given to techniques for finding out whether a measure is stable and valid?

a)Feasibility study

b)Psychometrics

c)Expert panels

d)Quantitative methods

13. What is the main purpose of a feasibility study?

a) To ensure a measure is easy to complete

b) To check cost-effectiveness of a measure

c) To advertise your research

Further Reading

Evans J, Rose D, Flach C, Csipke E, Glossop H, McCrone P, Craig T, Wykes T: VOICE: Developing a new measure of service users' perceptions of inpatient care, using a participatory methodology. Journal of Mental Health 2012, 21(1):57-71.

Challenges to user-led research and user-

produced knowledge

Frank Scepticism

Peter Tyrer, past editor of the British Journal of Psychiatry, writes:

“The engine of user involvement, while welcome in principle,……….may drive mental health research into the sand.”

Power Most of the projects we have been involved with are

‘collaborative’

Nearly always headed up by professor(s) of psychiatry or psychology

Not just status or naked power although that exists– more subtle

Are you a researcher or are you a patient?

“I wonder what your diagnosis is, then.”

Undermining user-produced knowledge

Hierarchies of Evidence RCT is the ‘gold standard’

But RCTs are not neutral – outcome measures devised by clinicians

‘Expert opinion’ counts as evidence but only as the weakest form

Experts are psychiatrists

Users as experts – a different knowledge perspective

The Cochrane Hierarchy needs revisiting

The charge of bias 1 Said, mostly implicitly, that user-research is

biased, anecdotal and carried out by people who are over-involved – ENMESH conference

We make no pretence of neutrality But all research comes from a certain standpoint Conventional researchers think what they do is

‘obvious’ – nothing is obvious User researchers more explicit about this than

mainstream researchers In my opinion the word ‘bias’ should be banished

from research discourse and all researchers should clearly say where they are coming from.

Bias 2 Although implicit some seem to think

that irrational people (the mad) cannot engage in the supremely rational activity of science

We epitomise ‘unreason’ (Foucault) So more difficult to find legitimacy and

credibility than consumer researchers in other medical disciplines

Paradox – user research in mental health ahead of that in other specialties

Some answers?

A new epistemology for user-led research – learning from feminism

‘Timeless oppositions’: Reason / unreason Culture / nature Intellect / emotion

First are male attributes and valorised So have women been excluded from

science and science not attentive to the concerns of women

Timeless oppositions and madness

The mad positioned as nature, emotion and crucially unreason

The Enlightenment valorised reason and so positioned the mad as its antithesis: Unable to reason Unable to labour

The Great Confinement

Standpoint epistemology and strong objectivity Standpoint epistemology has been seen

as essentialist Sandra Harding and ‘strong objectivity’ We have access to two discourses

Our own That of conventional science

A more complete picture – even a challenging one

Implications

User-led research can add to the jig saw or it can challenge conventional research

But needs to critique the methodologies and epistemologies of conventional research to do this successfully

Other ways service users may become involved in research Everybody in SURE employee of IoP BRC Service User Advisory Group (SUAG) Reference groups for projects e.g. The

Interface Study Sitting on Steering Groups Problem of tokenism although in SURE

also Should be parity of esteem and payment Training resources for service users

INVOLVE (NIHR) Continuum

Consultation Collaboration User-control

Researcher-initiated

Collaboration split Jointly initiated User-initiated

Activity14. The Cochrane Collaboration is associated with which type of evidence?

a)Systematic reviews

b)Focus groups

c)Ethnography

d)Service user research

15. Which two of the following statements are true?

a)RCTs are neutral

b)RCTs are considered the ‘gold standard’ of evidence

c)Service user research is neutral

d)User-led research can reduce the power imbalance in the research process.

16. Sandra Harding used which term to describe research by groups traditionally excluded from knowledge production?

a)Enlightenment

b)Strong objectivity

c)Unreason

d)Evidence-based medicine

17. Which is NOT a level of user engagement on the INVOLVE continuum?

a)Consultation

b)Collaboration

c)Facilitation

d)User-control

Further Reading

Sweeney A, Beresford P, Faulkner A, Nettle M, Rose D (eds.): This is Survivor Research. Ross-on-Wye: PCCS Books; 2009.

Activity Answers

Q. ANSWER(S) Q. ANSWER(S)

1. c 10. b

2. b 11. c

3. c 12. b

4. a 13. a

5. c 14. a

6. b 15. b, d

7. a 16. b

8. a, d 17. c

9. d