- Improving Lives Through Research -

Using mixed-method evaluation to improve practices for people living in the

community with mental health issues

Mike Chase

Presentation Outline• Outline the context and rationale for an evaluation

of a community psychiatric social inclusion day service

• Summarise the methodology, main research findings and the research impact and outputs

• Outline my recent efforts to initiate a similar impact model more local to UEL

• Open up for discussion the opportunities for and barriers to extending these types of Knowledge Exchange initiatives

Context and Rationale

• Individuals living with mental health difficulties remain among the most excluded people in society (Huxley & Thornicroft, 2003) 

• A two pronged attack...– Challenged societal attitudes, e.g., Time-to-change

programme (British Journal of Psychiatry, 2013)• Challenging stigma (

http://www.time-to-change.org.uk/)– Mental health day services are encouraged to

‘modernise’ (Social Exclusion Unit, 2004; Social Inclusion Programme, 2009).

Context and Rationale

• Modernising day services: the transition from bonding to bridging

– bonding: ‘drop-in’ type day services – bridging: emphasising ‘activity’ and ‘outward

focus’ towards recovery and wider inclusion • Place emphasis on social inclusion outcomes, e.g.,

community participation, employment and education and other ‘mainstream’ achievements

Context and Rationale

• Bridging day services nationally prevalent and extend across health and social care groups– Research on efficacy and how service users and

providers have experienced and responded to the transition is sparse (Social Inclusion Programme, 2009)

• Project aims: A mixed-method evaluation of a new social inclusion initiative for people living with severe and enduring mental illness in the UK – What did the service aim to achieve?– What did the service achieve and how was it

experienced?

Context and RationaleInitiating the project• Approached local health and social care commissioner

and provider with an offer of quid pro quo knowledge exchange

• Independent (mixed-method) evaluation of service – Output of an internal report with recommendations

• We learn how policy is translated into practice• Partnership built on existing collaborative links

• Service-user group research initiatives – Involvement Matters: A survey of service-user

participation in local mental health provision

The Wellbeing and Recovery Centre (WbRC)• Old traditional drop-in service closes June 2008

– A Cafe-type arrangement– Old building, compared to a dungeon

• Temporary premises until WbRC opened up in January 2009– Established voluntary sector organisation was service

provider– Premises afforded more light and aesthetically easier on the

eye • New WbRC aimed to:

– Emphasis on ‘activities’ to equip people in their ‘recovery’ and transition to more social inclusive environments

Context and Rationale

Context and RationaleTypes of new WbRC services• Sound-bites café (‘drop-in’ café)

F In local ‘mainstream’ community centreF Partly run by service-user volunteers

• Social inclusion and recovery servicesF Life and coping skills workshops (e.g., confidence

building, cooking, diet, relaxation and meditation)F Peer support groups, e.g., women's group, mixed groupsF Peer-led ‘mainstreaming’ initiatives, e.g., café, pub

meals and visitsF Art and ceramic type courses (in/ out house)F Recovery Star review (a holistic quality of life measure)

approx. every 3 months

Method• Mixed-method design (Stake, 1994) involving 3rd year

project student (CT) and myself (MC)• Participants and procedure

– 1:1 interviews (CT & MC; N= 17)• service users (n= 11, 7 males, 4 females), staff (n= 4, 2

managers and 2 support staff) and non-service users (n= 2, 1 male, 1 female)

– Participant observation (CT & MC; cafe and workshop observation and participation notes)

– Monthly attendance figures • No. of service-user attendees, types of workshops/

services they used and recovery star reviews– Survey questionnaire (N= 25)

Method• Analyses

– Thematic analyses (Braun & Clarke, 2006) of transcribed interviews and observation and participant notes (MC & CT)

– Quantitative (descriptive) analyses of service attendance data (April 2009 to March 2010 ) and survey results (MC)

– Regular meetings to share project and analyses progress with other research collaborators

Results

• What did the WbRC aim to achieve?

• What did the WbRC achieve and how was it experienced?

What did the WbRC aim to achieve?

‘... to give service users skills, back-to-work skills and social inclusion again, getting them back into the community, getting them used to working again, getting them in with people so they can make new friends.’

( Support Worker)‘Because people are often scared, ‘we don’t want to be part of

mainstream services, we don’t want to be out doing mainstream services, we want it as it is, it doesn’t feel safe to us, we want a safe place to come’. So that would be about very small steps of saying, you know like, ‘what if we arrange a small group, where you can all go down to a [city] Café, and let that start off and we back off?’

(General Manager)

Results

What did the WbRC achieve and how was it experienced?

• All service users interviewed valued and appreciated the service– A safe place to go (away from isolated

living environments) and to meet with others with shared experiences

– Appreciated the support of staff members

– Preferred the new building compared to the previous premises

Results

What did the WbRC achieve and how was it experienced?

Attendance figures

• From April 1st 2009 – March 31st 2010– Approx. 130 service users registered with WbRC– Averaged 51 service users per month

• 29 males, 22 females– Approx. 79 (61%) people not accessing services

Results

Café (day) Ceramics Café (evening)

Art Well be-ing & Rec

Relax & meditaion

Peer support

Woman's group

Others0

5

10

15

20

25Average no. of

attendees per session

Type of workshops/ services provided and average number of attendees (n= 51)

Type of workshop/ service

What did the WbRC achieve and how was it experienced?• Poor engagement with ‘bridging’ initiatives

– Predominantly same faces in different workshops– Peer-led mainstreaming initiatives sporadic (a lack of

interest or motivation)– No evidence that people took learnt knowledge and

skills home from workshops‘I might not use them that much at home because as soon as I get home I go to pieces and struggle to breath at home because I am paranoid about the neighbours downstairs and people outside.’ (Participant 3, Male, aged 45)

Results

What did the WbRC achieve and how was it experienced?– Lack of 1:1 intervention work

• ‘Star recovery’ reviews sporadic and non-integral to service provided– First six months (2 reviews/ month), second six

months (0.75 reviews/ month)– Interviewed service-user participants did not

remember what review was about or what it was used for

– Cf. Staff maintained they were administered every 3 - 6 months to each person

Results

What did the WbRC achieve and how was it experienced?– Lack of 1:1 intervention work

• 1:1 staff and service-user interactions were crisis driven

‘If people are feeling really unwell, if they are feeling they are reaching crisis point then we will offer key worker sessions, people can have a 1:1 with a staff member, where there is some extra support there and encourage people to come back to the sessions.’

(Participant 13, Support Worker)

‘They [staff] ask people if they’d like a chat if they see the need.’

(Participant 7, Male, aged 66)

Results

Discussion

• Bridging and Bonding: A Complex Story

General manager’s bridging perspective:

‘And then people are, ‘wow, there is this whole life out there that I didn’t know about and we don’t need you [staff members] to be a part of that, we always thought you had to be a part of that.’

DiscussionThe ‘reality’

– Little to no evidence people sought or relished mainstream social inclusion achievements

• Preferred company of others with shared difficulties and experiences away from the mainstream (i.e., bonding over bridging)

– But still active…• Gave people structure to their day• Meant people were less focussed on symptoms• Got people away from their isolated living

environments

Discussion

• Emphasis on prescribed social outcomes lead to a bludgeoning of ineffective services that missed the substance of people’s lives and their social inclusions needs- Regular “re-launching” of services to boost low

numbers- E.g., issues of loneliness, forming intimate

relationships and disclosure remained untouched by workshops

Discussion

• Large numbers of would-be service users (approx. 60%) were opting not to access the WbRC service – Reaction to service not being flexible or bridging

enough or to ‘modernisation’ and not enough bonding opportunities?

– Where do these people go and what do they do?

Discussion - Limitation

• Study introduced to participants (staff and service users) as an ‘informal service evaluation’ – Service providers interviewed may have

emphasized the ‘party line’ – Service users were keen to present the service

providers in a favorable light• ‘Agendas ’ evident in the data but mixed-method

analysis help us to delve beneath the surface

Report & Recommendations

Report Recommendations

• A need to ascertain what ‘social inclusion’ means to the individuals affected by severe and enduring mental illness– Placing more emphasis on positive (i.e., away from

crisis) one-to-one interventions– May or may not incorporate (re)entering more

‘mainstream’ environments – A need to revisit terms “person-centred” and “user-

centred” • More rhetorical statements than meaningful foundations

of interventions• Incorporate regular Recovery Star or similar validated

experience monitoring

Report Recommendations

• Follow-up research on to missing service users– Where do they go? What do they want from a

service?

• An adaptable and responsive service is needed to capture the complex relationship between bridging and bonding– Incl. outreach bridging support and more bonding

opportunity

– Incorporate new(ish) personalisation agenda

Other Research Outputs

• Chase, M., Thomas, C. & Costall, A. (in submission). ‘Bonding ‘or ‘bridging’? Implementing social inclusion in a contemporary psychiatric day service. Social Policy & Society.

Discussion Points

• The limitations of translating current ‘social inclusion’ policy into practice– Bridging day services make intuitive sense but

need to be grounded in the meaningful experiences of users of the provision

– Under the existing policy directives person-centered “choices” (Department of Health, 2005, 2006) for ‘bonding’ over ‘bridging’ count as ‘failures’

Discussion Points

“Although the mantra of ‘individual choice’ and ‘person centeredness’ often accompanies social inclusion initiatives, in effect the inclusion imperative inadvertently imposes certain choices as more desirable than others. In other words some choices are privileged and encouraged while others are problematised or pathologised.”

(Spandler, 2007)

Discussion Points• ‘Social inclusion’ should be based upon the

personal meaning that is placed on it– May not be immediate or obvious bridging capital – Failure to do this may result in further alienation

• Bridging and bonding is a complex relationship and not a dichotomy– Need to shift away from service-led outcome

interventions towards more pragmatic and sincere person-centred approaches

• More reflective, pragmatic and flexible approaches to rehabilitative interventions need to be considered and further researched (Greenwood & Levin, 2007; Schön, 1983)

Other Research Outputs

• Chase, M. (in submission). ‘The carrot and the state’: Personal experiences of shifting identities in the in the UK’s voluntary sector over two decades. Critical Social Policy.

A new local initiative: Similar KE model

• Context: Soup kitchen/ homeless drop-in East London• Service providers: St. Franciscan Friars• Aims: Independent (mixed-method) evaluation of service • Output: Internal report with recommendations for

change– Chase, M., Schneider, C. & Matoke, I. (in prep.). Lived

experiences of homeless people attending a soup kitchen in East London. Internal Report [due September, 2013].

• Other outputs– Watch this space...

• ‘Policy free’ environments• ‘Disenfranchised’ migrant Eastern Europeans

– Own and UEL student based initiatives and projects

Lessons learnt from these types of knowledge exchange

• Mixed-method and other types of service evaluations can be a mutually valuable process– Can positively influence lives and encourage a

reflective learning experience for all involved• Service commissioners and providers need to

be open to independent evaluation and change– Mixed-method analysis evaluation takes few

prisoners

Lessons learnt from these types of knowledge exchange

• We have a tremendous resource in the ability and diversity of our students

• If and at what point do quid pro quo KE/ KT initiatives become us our offering services on the cheap?– How can external funding opportunities be

developed?

Thank you!m.chase@uel.ac.uk

Questions?m.chase@uel.ac.uk