Evaluation of The Salvation Army’s Bridge Programme: 1 ...Evaluation of The Salvation Army’s Bridge Programme: 2. Treatment Outcomes 3. Evidence-based and Best Practice Guidelines

1. Developing a Partnership

Julien Gross, Emily Macleod, Linda Hobbs, Richard Egan, Claire Cameron,

and Tess Patterson

Evaluation of The Salvation Army’s Bridge

Programme:


Programme:

2.  Treatment Outcomes

3.  Evidence-based and Best Practice Guidelines

4.  Spirituality Matters

Introduction to the Project 2011 •  MOU with University of Otago

Introduction to the Project 2011 •  MOU with University of Otago

•  The Salvation Army wanted independent, evidence-based evaluation of its Social Programmes.

Evaluation: Why Do We Care?

Programme Provider Level • Allows programme providers to ‘see how

they’re doing.’

• Highlights areas of excellence … and areas for improvement.

• Guides possible future treatment directions.

Evaluation: Why Do We Care?

National Level •  Research and evaluation is critical for

informing the Sector, and for informing Government policy.

•  Research and evaluation leading to robust and evidence-based policy can make a difference to peoples' lives.

•  Funding ….

Introduction to the Project •  MOU with University of Otago (2011)

•  The Salvation Army wanted independent, evidence-based evaluation of its Social Programmes.

•  Starting point was an evaluation of the Bridge Programme Model of Treatment for people whose lives have been affected by their harmful use or, or dependence on, alcohol and/or drugs.

4 Cornerstones of The Bridge Model of Treatment

1.  The Salvation Army

2.  Partnership

3.  12 Step Recovery Journey

4.  Community Reinforcement Approach [CRA]

The Bridge Programme

• Nationwide AOD treatment service

•  15 AOD Addiction Centres

• Multiple options of delivery

-  Residential or community-based -  Intensive, full-time or day programme -  Outpatient service -  Social detox service

Introduction to the Evaluation 2012 •  Multidisciplinary team of researchers

Otago Research Team •  Dr Tess Patterson, Dip Clin Psych, Psychological Medicine

•  Dr Julien Gross, Psychology

•  Dr Emily Macleod, Dip Clin Psych, Psychological Medicine

•  Dr Richard Egan, Preventive & Social Medicine

•  Dr Claire Cameron, Biostatistician, Preventive & Social Medicine

•  Ms Linda Hobbs, Research Administrator, Psychological Medicine

Consultants:

•  Professor Andrew Bradstock, Centre for Public Theology (but now, Secretary for Church & Society, United Reformed Church, UK)

•  Dr Gavin Cape, Psychiatrist, DN Community Alcohol and Drug Services

Evaluation Research Team •  Tanja Ottaway Parkes, Phillipa Van Abs, Melinda Hayes

(Auckland) •  Keryn Roberts and Chloe Shadbolt (Waikato) •  Allanah Elvy‐Arnold, Paul Haycock, and Aggy Setefana‐Pue

(Wellington) •  Penelope Fleming and Shiona Morrison (Christchurch) •  Kieran Garner‐Hanson (Dunedin)

Introduction to the Evaluation 2012 •  Discussions about the scope

•  Translating evaluation objectives into research questions

Evaluation Objective Is the Bridge Programme Model of Treatment effective for clients and their family/whānau?

Research Questions 1.  Evaluate whether the Bridge Programme:

– Reduces or stops harmful substance use and improves real-world functional outcomes for clients.

– Measures up to national and international standards for substance abuse treatment approaches.

2.  Evaluate the role of spirituality

Introduction to the Evaluation 2013 – Development of the evaluation protocol,

selection of measures, training of Salvation Army research staff

– November, the evaluation kicked off at the Dunedin Bridge.

Research Components 1. Measure treatment outcomes for clients

attending the Bridge Programme;

2. Conduct a literature review of best practice guidelines;

3. Conduct a systematic review comparing Bridge Programme outcomes with other treatment programme outcomes;

4.  Evaluate the spirituality component of the Programme.


Programme:

2.  Treatment Outcomes

3.  Evidence-based and Best Practice Guidelines

4.  Spirituality Matters

More Information

• Access the full report online:

www.salvationarmy.org.nz/TestingTheBridge

•  http://www.mbie.govt.nz/about/whats-happening/2015-international-year-of-evaluation

Acknowlegements

– Clients of The Bridge Programme who generously agreed to participate in the programme evaluation

–  Staff and Directors of The Salvation Army Addiction Centres involved, particularly the amazing team of Research Assistants

– Our Departments and colleagues

Evaluation of the Salvation Army Bridge Programme: Treatment

Outcomes

Presentation by: Dr Tess Patterson

Evaluating the Bridge Progamme

•Measuring treatment outcome – Primary (substance use) – Secondary (consequential outcomes) - Changeable factors related to good

outcome

Method • 12 month prospective study • Psychometrically validated assessment

battery • Self report • Primary outcomes – Days of use – Severity of use

Measured • Secondary consequential outcomes – Health (physical and mental) – social status/ interpersonal problems – Quality of life – Employment status – Negative consequences – Criminal status

Measured • Changeable personal factors related

to good outcome – Motivation – Self efficacy – Locus of control

Also measured • Therapeutic alliance • Corroborative report of others •Questions – Participants views about the

programme

Methods • Addiction services – Auckland,

Manukau, Waitakere, Waikato, Wellington, Christchurch, Dunedin

• 478 clients invited • 382 consented to participate

Procedure • Invitation and informed consent • Full set of measures – Baseline – End of treatment – 3 month follow-up

Data analysis • All completed measures returned to

University of Otageo – Scored – Coded – Fidelity check

Participants • 382 participants between 20-to 73-

years of age • 65% male • 34% female

Results • Ethnicity – NZ European 66% – Maori 31% – Pacific Island 10% – other 9%

Results • Participants with problematic

alcohol use – AUDIT mean 25.47

• Participants with problematic drug use – DAST mean 15.24

• AUDIT and DAST scores indicate referral and treatment to

specialist treatment provider

Results

• Psychiatric status

– Patient Health Questionairre

•24% major depressive disorder

•14.2% other depressive disorder

•21.8% for panic syndrome

•16% met criteria for other anxiety

syndrome

– Addiction Severity Index (ASI)

psychiatric status

•0.31 (0=no problem, 1=extreme problem)

Completed treatment • 225 (69%) completed a therapeutic

dose of treatment • 100 (31%) did not complete a dose

of treatment •Drop out rate not uncommon – (e.g., drop out rates in AOD field range

21% to 67%) •Overall, good therapeutic alliance

(top quartile)

Results • Results presented – summary statistics (means, sd) – Number of analysis strategies

(confidence intervals, t-tests, correlations, linear mixed models)

• Results reported are on those who received a dose of treatment (n =225)

Primary Use •Q. Does the Bridge programme

reduce or stop harmful substance use?

Days of use

Figure 1. Mean DAYS OF USE for Alcohol, Cannabis, and Amphetamines at each time point.

Severity of use

Figure 2. Mean SEVERITY of alcohol use and drug use at baseline, end of treatment and follow-up. The data are shown collapsed across use classification.

Secondary Outcomes •Does the Bridge Programme

improve real world functional outcomes?

Physical Health

Figure 3. Physical health as measured by the WHOQoL-BREF physical health subscore at baseline, end of treatment, and follow-up. Note that an increase in score indicates an improvement in physical health.

Mental Health

Figure 5. Mental health as measured by the ASI psychiatric score at baseline, end of treatment, and follow-up. Note that a decrease in score indicates an improvement in medical health.

Family / Social Functioning

Figure 6. Mean social functioning as measured by ASI family score at baseline, end of treatment, and follow-up. Note that a decrease in score indicates an improvement in functioning.

Perceived Quality of Life

Figure 8. Quality of life as measured by the WHO QoL BREF Question 1 (life) at baseline, end of treatment, and follow-up. Note that an increase in score indicates an improvement in quality of life.

Employment Outcomes

Figure 9. Employment status as measured by the ASI employment composite (recent) score at baseline, end of treatment, and follow-up. Note that a decrease in score indicates an improvement in employment status.

Overall negative consequences

Figure 10. Mean overall CONSEQUENCES of alcohol use and drug use at baseline, end of treatment, and follow-up. The data are shown collapsed across use classification.

Criminality

Figure 11. Mean scores on the questions assessing participants’ criminal status at baseline, end of treatment, and follow-up.

Changeable personal factors

•Does the Bridge programme alter changeable personal factors related to good outcome?

Changeable personal factors

•Motivation increased • Self efficacy increased (for alcohol

and drug) • Locus of control shifted in a helpful

way (internal locus of control).

Discussion • All primary and consequential

outcomes improved •Many to a clinically and personally

salient level

Caveats • Results are on those who completed

a dose of treatment • End of treatment measures (76% ),

Follow-up measures (48%) • Follow-up time period 3 months

Overall conclusion • Evidence that the Bridge Programme

is effective

Assessment domains and measures used

Assessment Domain Measure Demographic information:

� Self-report

Identification of psychiatric status

� Online version of Addiction Severity Index (ASI-MV)- Psychiatric

Status sub-score � Patient Health Questionnaire (PHQ)

Identification of problematic substance use:

� Alcohol Use Disorders Identification Test (AUDIT) � Drug Abuse Screening Test (DAST)

Treatment goal:

� Self-report to single question, “During your time at the Bridge

Programme, what is your goal regarding your alcohol and/or drug use?”

Assessment Domain Measure

Substance use:

� ADOM Part A – questions regarding the type and frequency of

alcohol and other drug use

Severity of use:

� Online version of Addiction Severity Index (ASI-MV) - composite

(recent) score for alcohol and drug

Consequences of use:

� Drinker Inventory of Consequences –recent (DrInC) � Inventory of Drug Use Consequences – recent (InDUCdrug)

Social functioning:

� Online version of Addiction Severity Index (ASI-MV) -

Family/Social status composite (recent) sub-score � Drinker Inventory of Consequences –recent (DrInC) - Interpersonal

consequences sub-score � Inventory of Drug Use Consequences – recent (InDUCdrug) -

Interpersonal consequences sub-score

Physical health:

� Online version of Addiction Severity Index (ASI-MV) - Medical

status composite (recent) sub-score � World Health Organisation Quality of Life (WHO QoL) - Physical

Health sub-score

Mental health:

� Online version of Addiction Severity Index (ASI-MV)- Psychiatric

Status sub-score

Assessment Domain Measure

Perceived quality of life:

� World Health Organisation Quality of Life Assessment (WHOQoL-

BREF) - Question 1 – Life

Vocational functioning:

� Online version of Addiction Severity Index (ASI-MV) - Employment

status sub-score � Important People and Activities –‘ Work for pay’ and ‘work not for

pay’ hours per week1

Criminality:

� Drinker Inventory of Consequences –recent (DrInC) – Questions 41

& 42 � Inventory of Drug Use Consequences – recent (InDUCdrug) –

Questions 41 & 42

Activity:

� Important People and Activities - Non work related hours per

week1

Self-efficacy:

� Alcohol abstinence Self-Efficacy (AASES) � Drug Abstinence Self-Efficacy Scale (DASES)

Locus of control:

� Drinking related Internal-External Locus of Control Scale (DRIE) � Drug-related Locus of Control (DR-LOC)

[1] The Important People and Activities questionnaire was poorly completed so the data obtained from this measure was not included in the analysis.

Testing the Bridge: An Evaluation of the Effectiveness of

The Salvation Army’s Bridge Programme Model of Treatment

Evidence-based and best practice guidelines

Dr Emily MacleodDr Julien Gross

Dr Tess PattersonDr Richard Egan

Research Questions1. Evaluate whether the Bridge Programme:

– Reduces or stops harmful substance use and improves real-world functional outcomes for clients.

– Measures up to national and international standards for substance abuse treatment approaches•Literature Review of Best-Practice

Guidelines•Systematic Review of Evidence

Best Practice Guidelines:Literature Review

Multiple recommended approaches- Brief alcohol interventions- Less intensive alcohol interventions- Community Reinforcement Approach- Cognitive Behavioural Therapy- 12 Step Facilitation Approaches- Family Therapy- Pharmacotherapy


Bridge Programme consists of 4 components:

1. Partnership2. Community Reinforcement Approach

(CRA)3. 12 Step Recovery Journey4. The Salvation Army aspect


New Zealand - Māori and Pacific people over-represented

in AOD treatment- Important to recognise cultural values and

practices

Systematic Review

• To evaluate the effectiveness of the Bridge Programme Model of Treatment on key outcomes in comparison to other national or international alcohol and drug treatment programmes.

Systematic Review Method

• Inclusion criteria, e.g.,• clinical trials (RCT or CCT)• adult population• residential setting• psychosocial treatment component • relevant outcomes measured

• Searched a number of electronic sources (e.g., PubMed, CENTRAL, PsycInfo, EMBASE, CINAHL)

Interventions– Brief intervention (MI, information)– CBT– TSF– MET– Counselling– Couples therapy– Skills building – Adjunctive treatments (e.g., contingency

management, internet delivery, relapse prevention, reinforcement)

Results from Systematic Review

Primary outcome: Substance use- Use- Severity

Results:Internationally: A range of treatment approaches are effective Bridge Programme: Compares favourably to international treatment approaches

Systematic Review –Comparison of Data

Baseline End of treatment Follow-up

Salvation Army Evaluation

64.5% 99.0% 90.4%(3 month follow-up)

COMBINE (LoCastro et al., 2009)

25.3% 74.6% 67.8%(3 month follow-up)

62.7%(9 month follow-up)

Project MATCH 20% - 30% 80% - 90% 80% - 90%(15 month follow-up)

Example comparison:Alcohol use - percent days abstinent


Example comparison:Severity of alcohol use (ASI alcohol severity score)

z Baseline End of treatment Follow-up

Salvation Army Evaluation

Users of only alcohol:ASI: 0.51

Users of only alcohol:ASI: 0.16

Users of only alcohol:ASI: 0.15(3 month follow-up)

Greenfield et al. (2007) ASI: 0.44 – 0.45 ASI: 0.30 – 0.31 ASI: 0.22 – 0.41(3 month follow-up)

Results …

Secondary consequential outcomes

Results:Internationally: Mixed results. Treatment approaches did not always improve secondary outcomes. Bridge Programme: The Bridge Programme improved all of the secondary outcomes measured.


Example comparison:Secondary outcomes for drug users – overall negative consequences

(Inventory of Consequences for drug users)


Salvation Army Evaluation Users of single drug only (no alcohol):InDUC: 55.09

Users of single drug only (no alcohol):InDUC: 7.56*

Users of single drug only (no alcohol):InDUC: 6.30*(3 month follow-up)

Morgenstern et al. (2001)Morgenstern et al. (2003)

InDUC: 56.4 InDUC: 13.9* InDUC: 15.1*(6 month follow-up)

* significantly different to baseline


Example comparison:Mental Health(ASI vs SF-12)


Salvation Army Evaluation ASI Psychiatric: .31 ASI Psychiatric: .16* ASI Psychiatric: .15*(3 month follow-up)

Drummond et al. (2009) SF-12 Quality of Life:Mental Health: 46.6-49.2

SF-12 Quality of Life:Mental Health: 49.9-50.5(6 month follow-up)

* significantly different to baseline

Limitations & Conclusions• Difficult to truly compare to international

studies

• But overall– The Bridge Programme compares

favourably to international studies– The Bridge Programme adheres to

international best-practice recommendations

Evaluation of the Salvation Army Bridge Programme Model of

Treatment – spirituality findings Presentation by:

Dr Richard Egan, Dr Tess Patterson, Dr Emily Macleod, Dr Claire Cameron, Ms Linda Hobbs, Dr Julien Gross

University of Otago [email protected]

Outline

• Background • Research methods • Findings • Discussion

Spirituality matters •  Personal / professional

interest

•  Part of the whole – but reduced to ‘a part’ to examine, understand, develop ‘evidence’, improve care

•  Matters more at heightened times of life/illness

Our model of health/healthcare

matters bio-reductionist bio-psycho-social-spiritual

Whole person principle and approaches – holism, total care, Te Whare Tapa Wha, Fono Fale etc.

Spirituality in healthcare: Māori ContributionTe Whare Tapa Wha

Durie, M. 1985

Spirituality in healthcare: Māori Contribution

“Taha wairua is generally felt by Māori to be the most essential requirement for health”. (Durie, 1999)

“Without a spiritual

awareness and a mauri (spirit or vitality, sometimes called the life-force) an individual cannot be healthy… .” (Durie, 1999)

Spirituality in healthcare: the

zeitgeist Spirituality and Religion

•  Contested

•  Low attendance/rise of ‘nones’

•  Disentwining thesis:

growth of contemporary spiritualities

•  “I’m spiritual, not religious”

Reasonable evidence Meta-analysis x 3

Salsman, J. M., G. Fitchett, T. V. Merluzzi, A. C. Sherman and C. L. Park (2015). "Religion, spirituality, and health outcomes in cancer: A case for a meta-analytic investigation." Cancer: n/a-n/a.



In 2005, Stefanek et al’s review of the literature said: “The study of religion and spirituality and cancer is in its infancy. It is too early to determine what role the [R/S] constructs play in cancer outcome either related to the disease itself, or to quality of life and adjustment measures.” Salsman et al., 2015. p.2



[In 2015] the results confirm that R/S is significantly though modestly associated with patient reported mental, physical, and social health. Some. Park et al., 2015. p. 5




Well over a hundred measures of R/S have been used in research, and many have poor or unestablished psychometric properties. Salsman et al., 2015. p.3





Well over a hundred measures of R/S have been used in research, and many have poor or unestablished psychometric properties. Salsman et al., 2015. p.3

These results underscore the importance of attending to patients’ religious and spiritual needs as part of comprehensive cancer care. Jim et al., 2015. p. 1


What is spirituality? Map of the terrain.

Spirituality means different things to different people. It may include (a search for):

–  one’s ultimate beliefs and values;

–  a sense of meaning and purpose in life;

–  a sense of connectedness;

–  identity and awareness;

–  and for some people, religion.

It may be understood at an individual or population level.

Egan, R., R. MacLeod, C. Jaye, R. McGee, J. Baxter and P. Herbison (2011). "What is spirituality? Evidence from a New Zealand hospice study." Mortality 16(4): 307-324.

Some local spirituality studies – building evidence

•  Spirituality in New Zealand education* •  Spirituality in New Zealand hospice care* •  Psycho-social-spiritual supportive care in cancer* •  Spirituality in ODHB oncology ward •  Spirituality in medical education* •  Spirituality in aged care •  Renal specialists & spirituality* •  Spirituality and dementia study* •  Spiritual care professional development project* •  Spiritual care in cancer care across 16 countries •  Spirituality in NZ nursing care •  Spirituality in the Bridge Programme •  Funding applications in…

Spirituality background

•  Growing recognition of the power of spirituality-informed treatments for addiction.

“it is becoming increasingly difficult to exclude spirituality as a possible factor in the addiction recovery process for many individuals” (Stewart, 2008, p. 402)

•  Strong consumer desire

•  General underestimation

Spirituality

What is the role of spirituality in the Bridge Programme?

1)  Does the Bridge Programme increase spiritual well-being?

2)  Does spiritual well-being matter in terms of treatment processes and outcomes?

3)  Do spiritual practices and discussion of a higher power help in treatment processes?

Spirituality We used a mixed-method approach: Quantitative Data Collection •  World Health Organization Quality of Life

Assessment-Spiritual, Religious, or Personal Beliefs (WHOQoL-SRPB)

Qualitative Data Collection •  Thematic analysis of participants’ responses

to open-ended questions

Spirituality Results

Spirituality meant different things to different people:

Faith

Beliefs

Meaning

Purpose

Values

Figure 11. The percentage of participants selecting each definition.

Spirituality meant different things to different people:

But very few participants thought that

“it was meaningless.”

Figure 11. The percentage of participants selecting each definition.

Spirituality Results

Spirituality

Did the Programme increase spiritual well-being?

Figure 12. Overall level of spiritual beliefs at each time point. Note that an increase in score indicates an increase in spiritual beliefs.

Spirituality

Did particular components of spiritual beliefs change?

Spirituality

Did spiritual well-being matter in terms of treatment outcomes?

–  For users of alcohol, an increase in spiritual beliefs was associated with a decrease in severity of alcohol use.

–  Likewise for users of drugs.

Spirituality

At end of treatment, participants were asked a series of questions: •  Do you have a belief system that is important to you? Has this changed

since finishing the Programme? •  What matters to you most? Has this changed since finishing the

Programme? •  Did the Programme affect your spirituality or religious beliefs? •  Did the Programme alter any sense of meaning or purpose in your life? •  Are there spiritual practices (e.g. prayer, attendance at church) that have

helped you during your time in the Programme? •  How did the discussion about knowledge of a higher power affect your

experience in the Programme?

Spirituality

•  While only a minority, religious participants often framed their answers in terms of God, Jesus or other religious language.

•  Non-religious responses make up the majority

of answers across all questions.

•  What mattered most? - Key themes included sobriety, religious responses,

family/whanau, values, self-awareness, meaning & purpose, contribution, and ambivalence.

Spirituality

•  A small number of participants said that the Programme had no impact on their spirituality:

“No. Being abstinent has lifted the haze though” (144)

•  Some participants said that spirituality was important but that it had not changed over the Programme:

“always believed in Jesus and God” (152) ““My belief system has not changed..” (1409)

“My own values and morals” (1050)

Spirituality

Some participants said that spirituality was important and it had changed over the Programme:

“When I started (I had) no belief system. Now I believe in a higher power” (328)

“It made me feel like I could have a meaningful life without alcohol and drugs” (119)

“More, stronger feeling for family” (523)

“My sobriety is the most important thing to me. I have different priorities now. I was all about me when I was drinking” (1004)

Spirituality

•  Impact on spiritual beliefs

“Christian values have been awoken” (130) “It opened my eyes to my spirituality” (1050) “It gave me a better understanding of it (it’s not just about God.. It can be anything” (320) “yes it built more spirituality” and “it made me see better things in life than alcohol” (114)

Spirituality

Spiritual practices that helped:

•  Recovery Church

•  Prayer / serenity prayer

•  Meditation

•  Spirit Lifters

•  Karakia (only x 1)

Spirituality in healthcare : the

zeitgeist Spiritual Vacuum / Gap?

•  Growth of meaninglessness.

•  Materialism not enough?

•  Individual and Societal issue (a Public Health issue)

•  Re-emergence?



•  Growth of

meaninglessness.






•  Growth of

meaninglessness.




Some participants noted their awareness of a sense of meaninglessness or spiritual vacuum, either personally in society generally, “Greater awareness of spiritual bankruptcy. I now can say that I have a greater sense of enlightenment” (723) “I have found some of my soul” (310). “I realise there is a meaning and purpose to my life, before my life felt meaning[less]” (301) “Life is worth living” (1047).

NHS Scotland: spiritual care •  Spiritual care is usually

given in a one-to-one relationship, is completely person-centred and makes no assumptions about personal conviction or life orientation.

•  Religious care is given in the context of the shared religious beliefs, values, liturgies and lifestyle of a faith community.

Spiritual care: ethical Issues

Five Guidelines 1. Understand each patient’s spirituality 2. Follow patient’s wishes 3. Don’t impose spiritual care 4. Understand one’s own spirituality 5. Proceed with integrity.

(Winslow 2003)

Spirituality

Conclusions

•  Spirituality is a key component of the Bridge Programme. •  It is expressed through Recovery Church, prayer,

spirituality lifters/class, and the higher power component of the 12 Step programme.

•  Other, more subtle spiritual aspects of the programme

include a focus on meaning and purpose beyond addiction.

•  The research literature and the findings of the evaluation of the Bridge Programme suggest that facilitating spiritual well-being improves AOD outcomes.

•  Limitations/improvements

More Information

• Access the full report online:

www.salvationarmy.org.nz/TestingTheBridge

•  http://www.mbie.govt.nz/about/whats-happening/2015-international-year-of-evaluation

Acknowlegements

– Clients of The Bridge Programme who generously agreed to participate in the programme evaluation

–  Staff and Directors of The Salvation Army Addiction Centres involved, particularly the amazing team of Research Assistants

– Our Departments and colleagues

Comments or questions

‘Ko te Amorangi ki mua, ki te hapai o ki muri’

‘Place the things of the spirit to the fore,and all else shall follow behind’

Takitimu whakatauaki (proverb)

(Payne, Tankersley, & McNaughton A (Ed), 2003, p. 85)

THANK YOU

Documents

Evaluation of The Salvation Army’s Bridge Programme: 1 ...Evaluation of The Salvation Army’s Bridge Programme: 2. Treatment Outcomes 3. Evidence-based and Best Practice Guidelines