15

Click here to load reader

Monitoring Alliance and Outcome with Client Feedback Measures

Embed Size (px)

Citation preview

Page 1: Monitoring Alliance and Outcome with Client Feedback Measures

43 Journal of Mental Health Counseling

Volume 36/Number 1/January 2014/Pages 43–57

Monitoring Alliance and O

utcome

Monitoring Alliance and Outcome with Client Feedback Measures

Sidney L. Shaw Kirsten W. Murray

The therapeutic alliance is foundational to counseling practice and has amassed strong empir-ical support as being essential for successful counseling. Counselors generally rely on their own perspective when assessing the quality of the alliance, though the client’s perspective has been found to be a better predictor of outcome. Formal methods for eliciting client feedback about the alliance and counseling outcomes have been strongly supported in the literature, yet such limitations as time constraints hinder counselor efforts to gather formal client feedback. Two ultra-brief measures of alliance and outcome, the Session Rating Scale and the Outcome Rating Scale, are feasible methods for counselors to secure client feedback. This article reviews the two measures and makes a case for using empirical means to understand adult clients’ views of the therapeutic alliance.

A quality therapeutic relationship is the foundation of successful coun-seling. While far from a new concept, the idea that the relationship is central to human growth is increasingly relevant to the practice of counseling today (Bohart, 2003; Wampold, 2001). A common definition of the therapeutic alliance, on which we rely here, is that of Bordin (1979); it is an agreement between counselor and client on goals, treatment tasks or methods, and the relational bond. Meta-analytic research shows that alliance factors are major contributors to successful client outcomes (Wampold, 2001). Additionally, virtually all theoretical orientations acknowledge the importance of the alli-ance. However, an important nuance is that the client’s view of the alliance is a better predictor of client outcome than the counselor’s view (Bachelor & Horvath, 1999; Bedi, Davis, & Williams, 2005; Horvath & Bedi, 2002; Horvath & Symonds, 1991; Wampold, 2001). Further, counselor views of essential rela-tionship factors often do not correlate well with client views (Tyron, Blackwell, & Hammel, 2007). While steadfast counselor intentions to develop a therapeu-tic relationship are essential, the true litmus test of an effective alliance is how the client perceives it.

In everyday practice, counselors generally rely on their own conjectures about the therapeutic relationship, although these notions are often at odds

Sidney L. Shaw is affiliated with the University of Montana and Walden University, Kirsten W. Murray with the University of Montana. Correspondence about this article should be addressed to Sidney L. Shaw, College of Education and Human Sciences, Room 210, Department of Counselor Education and Supervision, 32 Campus Drive, University of Montana, Missoula, MT 59812. Email: [email protected].

Page 2: Monitoring Alliance and Outcome with Client Feedback Measures

44

with client views (Hannan et al., 2005; Stewart & Chambless, 2007). Because the client’s perception of the therapeutic alliance is a strong predictor of out-come, and a better predictor than the counselor’s perceptions, we recommend a formal model for collecting continuous feedback from the client. Further, formally assessing the counseling relationship becomes critical when honoring the therapeutic alliance and potentiating its powerful effects. Collecting feed-back from clients directly privileges their voice, attends to the client-counselor relationship, and establishes a framework for constructing individualized, cli-ent-directed counseling.

This article reviews evidence supporting the need for formally collecting alliance and outcome feedback from clients. The Outcome Rating Scale (ORS; Miller, Duncan, Brown, Sparks, & Claud, 2003) and Session Rating Scale (SRS; Duncan et al., 2003) are reviewed as evidence-based methods for formally monitoring alliance and outcome. These measures are introduced as part of the Partners for Change Outcome Management System (PCOMS; Duncan, 2012). We make the case for formal client feedback as an ethical obligation and present considerations for employing alliance and outcome feedback processes.

PredicTinG counselinG success

A large body of research, including exhaustive meta-analyses, has found the therapeutic relationship to be a strong, if not the best, predictor of success-ful counseling (Conners, Carroll, DiClemente, Longabaugh, & Donovon, 1997; Orlinsky, Rønnestad, & Willutzki, 2004; Wampold, 2001). Studies con-tinue to support the notion that the partnership between counselor and client is one of the best indicators of successful counseling outcomes in a variety of con-texts (Horvath & Symonds, 1991; Martin, Garske, & Davis, 2000). Further, the client’s rating of the alliance is generally a better predictor of a good counseling outcome than the counselor’s view (Bedi et al., 2005; Horvath & Bedi, 2002). Given the established potency of the counseling relationship as a determinant of outcome, monitoring the alliance from the client’s perspective is crucial to counseling success.

The Client’s PerspectiveResearch consistently demonstrates the powerful effect of the client’s

perception of a positive therapeutic relationship. Citing over 100 studies that elicit the client’s perspective, Norcross (2010) concluded that the relationship is central to therapeutic success. The shift from the counselor’s perspective of the therapeutic relationship to formally understanding the client’s perspective is strongly supported by research indicating that clients show significantly more therapeutic gain when their feedback is sought (Duncan, Miller, Wampold, & Hubble, 2010; Lambert, 2010).

Though apparently simple on the surface, the process of reliably and consis-tently obtaining the client’s perspective of relationship factors is subtle and more powerful than is immediately evident. Formal client feedback measures were

Page 3: Monitoring Alliance and Outcome with Client Feedback Measures

Monitoring Alliance and Outcome

45

examined in a study where 48 counselors (26 trainees and 22 licensed staff) in a “real-world” clinical setting were asked to predict their clients’ outcomes using their “clinical judgment” (Hannan et al., 2005). Client outcomes were then assessed using Outcome Questionnaire-45 (OQ-45), a reliable assessment for determining the client’s perspective on improvement or outcome (Miller et al., 2003). It was found that counselors’ perceptions of counseling outcome aligned poorly with those of clients. The study found that 40 clients showed significant decline or deterioration during counseling, and in only one case did counselors correctly identify a client as deteriorating. In other words, the counselors in the study did not identify the 39 out of 40 clients who were getting worse during counseling. These findings underscore the importance of counselors seeking to understand client perspectives to use formal processes as well as clinical judg-ment and general impressions (Hannan et al., 2005).

Greenberg, Watson, Elliot, & Bohart (2001) presented another finding that emphasized both the correlation of therapeutic relationship and outcome in counseling and the discrepancies between counselor and client percep-tions. Their meta-analysis of 47 studies comprising 3,026 clients examined the impact of counselor empathy, a central relationship factor, on outcome. They found that empathy was indeed a good predictor of outcome with a moderate effect size of .32. Further, the theoretical orientation of the counselor was not predictive of outcome. Perhaps most important, the ratings of empathy varied significantly depending on the evaluator (client, observer, or counselor). Client ratings of empathy (mean uncorrected r = .25) were found to be the best pre-dictor of outcome, followed by observer ratings (.23), and counselor ratings (.18)—the least predictive of the three (Greenberg et al., 2001).

The Client’s View of EmpathyNot surprisingly, the question of whether a counselor is being empathic

can only be answered by the client. Empathy, a central component of ther-apeutic relationships, is classically defined by Carl Rogers (1957) as “the therapist’s sensitive ability and willingness to understand clients’ thoughts, feelings, and struggles from their point of view” (p. 98). Numerous studies have underscored the importance of the client’s view of empathy and the relation-ship. For instance, client-perceived empathy has repeatedly been found to be a better predictor of outcome than counselor-rated empathy (Barrett-Lennard, 1981; Bohart, Elliott, Greenberg, & Watson, 2002; Gurman, 1977; Orlinsky, Grawe, & Parks, 1994); and meta-analytic research consistently finds that the client’s view of relationship factors is more predictive of a successful counseling outcome than the counselor’s view (Bachelor & Horvath, 1999; Horvath & Bedi, 2002; Horvath & Symonds, 1991; Wampold, 2001). It is thus evident that understanding the therapeutic relationship requires feedback from the client. The essence of empathy, in the practical sense, is the counselor’s efforts to be empathic, but more important from both an empirical and clinical standpoint is the client’s sense of experiencing empathy. Though counselors can make efforts to respond in ways they think are empathic, whether they are being experienced as empathic requires continuous client feedback.

Page 4: Monitoring Alliance and Outcome with Client Feedback Measures

46

The cAse for formAl clienT feedBAck

Collecting feedback from the client emphasizes counseling tenets related to understanding clients’ subjective experiences, cultivating a quality relation-ship, responding with empathy, and supporting clients’ abilities to choose their goals and how to meet them (Cain, 2001; Raskin, Rogers, & Witty, 2008). Research on formal client feedback has burgeoned in recent years, and use of formal feedback measures has been found to consistently improve client out-comes (Duncan et al., 2010; Lambert, 2010). For instance, a randomized clin-ical trial that covered 110 community mental health clients and 28 counselor trainees found that client feedback about the alliance and outcome increased client outcome in comparison to the no-feedback group (Reese et al., 2009). This study found that the mean effect size for counselor trainees utilizing client feedback was .92, but trainees in the no-feedback condition had a mean effect size of .23 (Reese et al., 2009).

A randomized clinical trial of couple counseling randomly assigned cli-ents to a feedback group (i.e., the counselor would obtain session-by-session feedback from clients using an alliance measure and an outcome measure) or treatment as usual group (Anker, Duncan, & Sparks, 2009). Pre-study surveys of the counselors showed that they all believed they had already acquired out-come and alliance feedback from their clients without the use of a formal feed-back process and that formal feedback would not enhance their effectiveness. yet nine out of the 10 counselors improved their client outcomes when they sought formal client feedback using the ORS and the SRS (Anker et al., 2009). This finding illustrates not only the benefits of feedback but also the tendency for counselors to assume that an informal method of checking-in with clients is adequate.

In addition to randomized clinical trials, evidence supporting the use of client-driven formal feedback has also emerged from quasi-experimental research in clinical settings. Miller, Duncan, Sorrell, and Brown (2005) exam-ined counseling outcomes when 6,424 culturally and economically diverse clients provided feedback to counselors using the ORS and SRS. Researchers obtained baseline measures of client outcomes over a six-month period during which counselors were not exposed to formal client feedback. Following the baseline period, counselors gathered session-by-session feedback from clients about their view of the therapeutic alliance and outcome of counseling using ORS and SRS client report measures at every session. The ORS and SRS opened dialogue between clients and counselors about the client’s experience of therapeutic change and the alliance; simply providing counselors with ses-sion-by-session feedback about the client’s view of the alliance and outcome doubled the effect size from the baseline phase. The act of consistently engag-ing with clients about their experience of the relationship and the degree to which the sessions were helpful had a profound influence on client outcome.

Page 5: Monitoring Alliance and Outcome with Client Feedback Measures

Monitoring Alliance and Outcome

47

Improving Alliance and OutcomeThe practice of integrating client feedback requires counselors to step

out of their own subjective view and check their perceptions with those of the client. The process of formally collecting feedback from the client has been shown to improve both alliance and outcome (Duncan et al., 2010). Seeking formal feedback from clients about the therapeutic relationship becomes central if counselor attempts to build an alliance are to be successful. Further, when clients have visual markers of their positive counseling outcomes (based on the feedback they give), early progress becomes concrete and is easily rec-ognized. This recognition of progress supports even more positive outcomes, encouraging clients to continue to make gains.

The importance of consistently monitoring client outcomes is highlighted by numerous studies finding that early client improvement predicts a good out-come at the end of counseling and that the majority of change occurs early in treatment (Baldwin, Berkeljon, Atkins, Olsen, & Nielsen, 2009; Brown, Dreis, & Nace, 1999; Howard, Kopta, Krause, & Orlinsky, 1986). Without collecting feedback from clients, counselors are left to make assumptions about client trajectories of change. The beneficial effect of alliance and outcome feedback is clearly linked to removing guesswork and obtaining clarity about predictors of positive client outcomes. However, the question remains: if seeking formal client feedback is central to a strong alliance and positive outcomes, why is it not common practice?

Barriers to Collecting FeedbackUnfortunately, using alliance and outcome measures can trigger resis-

tance in counselors, perhaps partly because many measures are too time-con-suming for use in everyday practice (Miller, Duncan, & Hubble, 2004). Some common alliance measures range from 12 to 19 items and measures like the Outcome Questionnaire consist of 30 to 45 items (Duncan et al., 2003). Such lengthy measures work well for research purposes, but in clinical settings their demands on time and energy interfere with their consistent use. In a study of formal feedback using more lengthy measures, 75% of counselors stopped gathering client feedback because the measures were cumbersome and lengthy (Miller et al., 2004).

In addition to time demands as barriers, some counselors may be opposed for philosophical reasons. For instance, the idea of formally assessing some-thing as personal as the therapeutic relationship may seem to contradict the value of holism rather than reductionism (Raskin et al., 2008). Further, rating an alliance numerically runs the risk of mechanically diluting empathy to a number on a page. Lastly, such formal assessments have the potential to promote a directive stance and endorse a counselor’s power. In light of these philosophical concerns, we will explore a more nuanced understanding of the process for collecting formal client feedback while continuing to support the case of utilizing focused, time-sensitive formal client feedback tools to empower clients and improve counseling outcomes.

Page 6: Monitoring Alliance and Outcome with Client Feedback Measures

48

oBTAininG formAl clienT feedBAck

Used in counseling settings, alliance and outcome measures are intended to improve services by exposing counselors to client feedback that will allow them to individualize and tailor their services (Smith, Fischer, Nordquist, Mosley, & Leadbetter, 1997). These measures also help counselors ensure that the flow of the session does not influence whether or not they check in with the client about the relationship, and a norm is developed for collecting feedback. In particular, when counselors commit to using brief alliance and outcome measures, the formal process serves as a safeguard to push counselors to rou-tinely engage clients about their perspective rather than relying on their own suppositions about the client’s experience. Just as counselors have rituals for scheduling appointments and taking session notes, they can also incorporate brief measures into sessions to encourage client feedback and dialogue about the alliance and outcome.

In addition to tailoring and individualizing services, thoughtful presen-tation of formal client feedback methods in itself delivers a message to clients that the counselor intends to form collaborative, empowering relationships with them and that their perspective is consistently valued as a guiding force in counseling. These routine inquiries support frequent and consistent means for understanding alliance and outcome in a way that honors the client’s voice and creates a context for the counselor to accept influences to improve and tailor services. Finally, a number of studies have found that formal client feedback reduces drop-out rates and improves counseling outcomes (Duncan et al., 2010; Hatcher & Barends, 1996; Miller et al., 2005). Given these benefits and the intrinsic therapeutic value of honoring client feedback, attending to client views of the counseling alliance and outcomes is essential to support and priv-ilege the client’s experience. We recommend adopting a norm of consistently requesting formal client feedback, and using such formal measurement tools for collecting that feedback as the SRS and the ORS.

The SRS and the ORSThe SRS (Duncan et al., 2003) and ORS (Miller et al., 2003) were devel-

oped to fulfill a need for ultra-brief tools to regularly gather client feedback in counseling settings. Their systematic, session-by-session use falls under the evidence-based treatment program of the Partners for Change Outcome Management System (PCOMS). PCOMS is a client feedback system that was established as an evidence-based practice in January 2013 by the Substance Abuse and Mental Health Services Administration (SAMHSA) National Registry of Evidence-based Programs and Practices (NREPP). The PCOMS can be used with or integrated into any counseling theory or method (Duncan, 2012). The PCOMS clinical process involves tracking client reports of alliance and outcome and modifying treatment in response. Collecting formal feedback from clients about the alliance and outcome is intended to improve client outcomes and identify clients who may not be improving with expected trajec-tories of change (Duncan, 2012).

Page 7: Monitoring Alliance and Outcome with Client Feedback Measures

Monitoring Alliance and Outcome

49

The brevity and simple administration of the SRS and ORS, each of which has four items rated on a visual analog scale, make them highly suit-able to clinical practice. They demonstrate strong internal consistency and good concurrent validity with lengthier measures of alliance and outcome (Campbell & Hemsley, 2009). The two scales can be easily administered every session, taking just two minutes to administer and score. In addition, ultra-brief analog scales such as the SRS and ORS generally have more face validity than longer measures, which can be remote from the client’s experience (Duncan et al., 2003; Miller et al., 2003). The client is instructed to simply place a hash mark on the analog lines, indicating via the SRS their experiences of alliance and via the ORS their level of distress in four domains of their lives (Duncan et al., 2003; Miller et al., 2003). Since each analog line on the SRS and ORS is 10 cm long, the SRS and the ORS both have a maximum score of 40. The counselor measures client responses at every session, using a metric ruler to score them (Duncan et al., 2003; Miller et al., 2003). This gives counselors the client’s perspective of alliance and outcome at every session and also the patterns across sessions. As a result, counselors can track client progress over time but also address alliance or outcome issues in the moment as they arise.

The four scales of the SRS (Appendix A) measure the client’s view of the therapeutic alliance for a particular session in terms of the therapeutic relationship (degree to which the client felt heard, understood, and respected); the goals and topics (degree to which the client felt that the session focused on the goals he or she wanted); the approach or method (degree to which the counselor’s approach in the session was a good fit); and overall (degree to which the session was “right” for the client). The client completes the SRS near the end of each session. The counselor reviews the responses and opens discussion with the client about them. This immediate feedback thus gives counselors the opportunity to address alliance problems in the moment. The SRS typically takes about one minute to administer and score and is available in 25 languages. A child version is recommended for use with children aged 6 to 12 (Miller et al., 2004). Used appropriately, the SRS facilitates open dia-logue about the client’s experience of the session and the alliance. The SRS has demonstrated strong internal consistency (α = .88), test-retest reliability of .64, and validity coefficient correlations with the Working Alliance Inventory that range from .37 to .63 (Campbell & Hemsley, 2009).

The four scales of the ORS (Appendix B) measure the client’s view of change relative to the initial complaint from the first session and across all sessions. The four scales measure views individually (personal well-being); interpersonally (family, close relationships); socially (work, school, friendships); and overall (general sense of well-being). The individual, interpersonal, and social scales are adapted from the three domains of functioning assessed by the OQ-45.2 (Miller et al., 2003). Unlike the SRS, the ORS is designed to be administered at the beginning of each session to find out how the client has been doing since the last session. When the ORS is completed, the counselor scores and discusses the responses with the client. As with the SRS, the ORS takes about one minute to administer and score, is available in 25 languages

Page 8: Monitoring Alliance and Outcome with Client Feedback Measures

50

and in a child version, and provides counselors with real-time feedback about the client’s view of progress or lack thereof (Miller et al., 2004). The ORS has demonstrated strong internal consistency (α = .87 - .96) and a respectable cor-relation coefficient (r = .59) with the much lengthier, well-validated OQ-45.2 (Campbell & Hemsley, 2009). The ORS correlation with the OQ-45.2 pro-vides evidence of its construct validity (Reese, Norsworthy, & Rowlands, 2009).

Counselors use of the SRS and ORS is quite high in comparison to their longer counterparts. For instance, when use of the ORS and OQ-45 were com-pared over a 12-month period at a community family service agency, the ORS had an 89% compliance rate and the OQ-45 a 25% compliance rate (Miller et al., 2003). Similarly, the SRS had a 96% compliance rate and the 12-item Working Alliance Inventory a 29% compliance rate (Duncan et al., 2003). These compliance rates, along with their validity and reliability, make the SRS and ORS well-suited to counseling practice. The practicality of the SRS and ORS thus makes them a feasible option for counselors wishing to honor the client’s voice and improve client outcomes via client feedback. Integration of the SRS and ORS into daily practice also communicates to clients the impor-tance of their voice in dialog about two factors of critical importance to clients and counselors, the alliance and the outcome.

Using the Measures: A Brief IllustrationThe SRS and ORS are practical because they are brief, valid, and pro-

mote dialogue with clients about important aspects of treatment. Because their utility hinges on competent, client-directed integration, the following script is provided as guidance:

Counselor: I have a couple of short forms that I ask clients to com-plete, one near the beginning of each session and one near the end. These forms help us keep track of how things are improving or not improving in your life and also how our sessions are going. Does that sound okay?

Client: Sure.

Counselor [giving the client the ORS]: This one focuses on import-ant areas of your life that could improve. I’ll keep track of these over time so we can see how things are changing or not changing. Since this is our first meeting, today’s score will rate how things have been going up until now. Just put a mark on each of these lines showing how things are going in these four areas of your life, lower scores to the left and higher scores to the right. Make sense?

Client: Sounds good to me. [Client completes the ORS.]

The counselor then scores the ORS using a metric ruler and can use the ORS score to open the session by simply acknowledging areas the client scored as particularly high or low. Opening these dialogues may sound something like, “I noticed you scored this section lower than the others. Can you tell me more

Page 9: Monitoring Alliance and Outcome with Client Feedback Measures

Monitoring Alliance and Outcome

51

about what is most troublesome here?” or “This area continues to be highly rated for you. Tell me more about what is going well here.” As the session unfolds it is important to connect client presenting problems to scores on the ORS.

Near the end of each session the counselor engages the client about the alliance via the SRS:

Counselor: I’d like to ask you to fill out one more brief form before we end our session today. This one is to help us understand how our sessions might be adjusted to improve our work together. I really want any feedback you have. you should know that I’m not looking for a perfect score. I know that life isn’t perfect and neither am I. This will help me adjust things here to make sure you get the most from our time together, so I’d appreciate any feedback you can give about today’s session. If something wasn’t quite right for you I won’t take it personally and I can handle the feedback. Like the first form, this one has four scales. Make a mark on each line showing how you felt about today’s session. Again, mark lower scores toward the left and higher scores toward the right. Also, like the first form, we will spend a little time talking about your ratings. Any questions?

Client: No. Seems pretty clear. (Client completes the SRS.)

Counselor: Thanks for taking time to think about each of these scales before marking down a score [counselor acknowledges the client’s thoughtfulness and care in completing the form]. How you feel about our sessions is really important to me. [Counselor quickly scores the SRS.] I notice that you scored the “goals and topics” scale a bit lower than the others. Can you tell me a little more about what wasn’t quite right there?

Client: Well, you listened really well and I felt understood by you. I felt like I had so much to talk about with problems at my job that I just didn’t get around to discussing my boyfriend. We’re arguing a lot.

Counselor: Thanks so much for telling me that and I’m glad you felt understood and listened to today. I want to make sure that we are focusing on topics that are really important to discuss, so I’ll make a note about your relationship with your boyfriend.

Client: Great. It’s not a real big deal but our arguing is putting a strain on us and me.

Counselor: Sure, and that strain can impact so much of what’s going on for you. I’m glad you feel comfortable bringing this up and that you can tell me where we should focus.

As indicated in these examples, the ORS and SRS are tools for dialog as well as for tracking progress over time. After several sessions, counselors can

Page 10: Monitoring Alliance and Outcome with Client Feedback Measures

52

construct a simple line graph of client ORS scores and also use this to guide dialog with clients. Having a visual representation of client ORS scores over time is useful so that counselors and clients can work together to tailor treat-ment to maximize outcomes. In addition, as indicated in the example, the SRS can give clients an opportunity to voice areas of concern about a session. In this case, the counselor gained understanding of what was not discussed in the ses-sion that the client may have simply forgotten to mention, did not have enough time to talk about, or was possibly avoiding. Ultimately, the SRS and ORS help counselors and clients collaboratively track alliance and outcome while also opening dialogue about concerns that might otherwise remain unexpressed.

limiTATions And concerns

Although it is clear that the potential for therapeutic success can be fos-tered with formalized client feedback models like the SRS and ORS, there are also possible limitations. As is true with the art and science of assessment, the power of the instrument rests in how it is used. If not protected against, three risks may surface when incorporating formal client feedback: a mechanistic counseling process, a propagation of power over the client, and a reductionist understanding of the therapeutic alliance.

Central to counseling is being present in the moment and establishing a rich and meaningful connection with the client. The basis for this relationship is belief in a client’s potential to grow with the necessary conditions. With this tenet in mind, it is necessary for a counselor to establish a therapeutic environ-ment, and then adjust to client preferences and needs. However, perfunctory use of feedback measures can dilute the richness of dialogue and client-direct-edness that stems from thoughtful integration of formal client feedback. The ORS and SRS are not meant to be rushed bookends to a session but to be clin-ical tools for sparking deeper discussions of the alliance, monitoring change, and individualizing treatment. If circumstances are hurried, the quantitative indicators may lose their utility because the counselor is using them mechani-cally. Done well, however, integration of formal client feedback not only honors the ethical mandate to monitor counselor effectiveness (American Counseling Association, 2005; American Mental Health Counselors Association, 2010) but also helps to elevate the client’s voice in relation to the direction and effective-ness of the counseling experience.

Formal feedback must be integrated into the counseling process with grace and transparency, allowing clients to recognize their freedom in a session and the counselor to maintain a nonhierarchical role. Done with sensitivity and intent to honor the client’s voice, formal feedback is formative as well as summative: The scores on the page push counselors to engage in client-driven dialogue in order to individualize counseling (and challenge counselor assumptions).

Fostering client empowerment, which is at the root of counseling, can be cultivated with thoughtful integration of formal client feedback. However, a contradiction emerges in the double bind of privileging the client’s voice

Page 11: Monitoring Alliance and Outcome with Client Feedback Measures

Monitoring Alliance and Outcome

53

while also directing the client to quantify the counseling experience in a coun-selor-determined format. For example, if participation in the feedback process results in forcing scales upon clients and heavily influencing their responses, the client’s voice is lost, and so is the I-thou relationship. To best protect against this power-broking stance, the counselor must ensure there is clear informed consent when asking the client to participate in the formal feedback process and communicate and behave in a way that privileges the client’s perspective. When clients perceive that the counselor embraces the formal feedback pro-cess with humility and openness, they become the primary guides in tailoring services to best meet their needs.

A final critique of the formal client feedback process surfaces when the view of the therapeutic alliance is reductionist. For example, when referencing the SRS, it could be argued that the scale limits client response by ground-ing their reactions in a few specific questions and analog ratings. The risk of reducing the therapeutic relationship to four questions is especially high when a counselor embraces the quantitative data as the endpoint. Rather, we recom-mend embracing the scale as a gateway for client feedback and using the scales and results to open dialogue. Ultimately, it is vital that a process orientation be embraced when discovering the nuances of the therapeutic relationship. A solid therapeutic relationship is not something arrived at in accordance with four quantifiable domains but is rather something the counselor becomes part of by constantly evaluating, learning, and responding.

conclusion

The therapeutic relationship is at the crux of successful counseling (Norcross, 2010; Wampold, 2001). Although the counseling relationship is the most potent therapeutic factor, it has been found that counselors are not as accurate as they believe at understanding client perspectives of empathy and the relationship (Greenberg et al., 2001; Orlinsky et al., 1994). Failure to seek client feedback leads to a breakdown in understanding the client’s subjective experience and promoting client empowerment. Because the client perspec-tive is central to a strong alliance, client feedback must be sought routinely. yet counselors often assume that they do check in with clients about their subjective experience and that formal feedback from clients will not improve outcomes (Anker et al., 2009).

The SRS and ORS invite client feedback consistently, encouraging coun-selors to actively understand the client’s perspective, and empowering clients to tailor their counseling experience. However, any formal feedback process is only as useful as the way it is presented and used; counselors are cautioned to avoid mechanistic, reductionist, and power-propagating stances. To privilege the client’s voice, research clearly reveals that a formal means of collecting cli-ent feedback is necessary so that counselors can step beyond assumptions about alliance and outcome, begin to honor the client’s perspective, and maximize client benefit.

Page 12: Monitoring Alliance and Outcome with Client Feedback Measures

54

references

American Counseling Association. (2005). ACA code of ethics. Alexandria, VA: Author.American Mental Health Counselors Association. (2010). American Mental Health Counselors

Association code of ethics. Retrieved from http://www.amhca.org/assets/content/AMHCA_Code_of_Ethics_2010_update_1-20-13_COVER.pdf

Anker, M. G., Duncan, B. L., & Sparks, J. A. (2009). Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77, 693–704. doi:10.1037/a0016062

Bachelor, A., & Horvath, A. (1999). The therapeutic relationship. In M. A. Hubble, B. L. Duncan, & S. D. Miller (Eds.), The heart and soul of change: What works in therapy (pp. 133–178). Washington, DC: American Psychological Association.

Baldwin, S. A., Berkeljon, A., Atkins, D. C., Olsen, J. A., & Nielsen, S. L. (2009). Rates of change in naturalistic psychotherapy: Contrasting dose–effect and good-enough level models of change. Journal of Consulting and Clinical Psychology, 77, 203–211. doi:http://dx.doi.org/10.1037/a0015235

Barrett-Lennard, G. T. (1981). The empathy cycle: Refinement of a nuclear concept. Journal of Counseling Psychology, 28, 91–100. doi:10.1037/0022-0167.28.2.91

Bedi, R. P., Davis, M. D., & Williams, M. (2005). Critical incidents in the formation of the therapeutic alliance from the client’s perspective. Psychotherapy: Theory, research, practice, training, 42, 311–323. doi:10.1037/0033-3204.42.3.311

Bohart, A. C. (2003). Person-centered psychotherapy. In A. S. Gurman & S. B. Messer (Eds.), Essential psychotherapies: Theory and practice (2nd ed., pp. 107–148). New york, Ny: Guilford Press.

Bohart, A. C., Elliott, R., Greenberg, L. S., & Watson, J. C. (2002). Empathy. In J. Norcross (Ed.), Psychotherapy relationships that work (pp. 89–108). New york, Ny: Oxford University Press.

Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice, 16, 252–260. doi:10.1037/h0085885

Brown, J., Dreis, S., & Nace, D. (1999). What really makes a difference in psychotherapy outcomes? Why does managed care want to know? In M. Hubble, B. Duncan, & S. Miller (Eds.), The heart and soul of change (pp. 389–406). Washington, DC: American Psychological Association.

Cain, D. J. (2001). Defining characteristics, history, and evolution of humanistic psychotherapies. In D. J. Cain & J. Seeman (Eds.), Humanistic psychotherapies: Handbook of research and practice (pp. 3–54). Washington, DC: American Psychological Association.

Campbell, A., & Hemsley, S. (2009). Outcome Rating Scale and Session Rating Scale in psychological practice: Clinical utility of ultra-brief measures. Clinical Psychologist, 13, 1–9. doi:10.1080/1328420080267639

Connors, G. J., Carroll, K. M., DiClemente, C. C., Longabaugh, R., & Donovan, D. M. (1997). The therapeutic alliance and its relationship to alcoholism treatment participation and outcome. Journal of Consulting and Clinical Psychology, 65, 588–598. doi:10.1037/0022-006X.65.4.588

Duncan, B. L. (2012). The Partners for Change Outcome Management System (PCOMS): The heart and soul of change project. Canadian Psychology/Psychologie Canadienne, 53, 93–104. doi:10.1037/a0027762

Duncan, B. L., Miller, S. D., Sparks, J. A., Claud, D. A., Reynolds, L. R., & Brown, J. (2003). The Session Rating Scale: Preliminary psychometric properties of a working alliance measure. Journal of Brief Therapy, 3, 3–12.

Duncan, B. L., Miller, S. D., Wampold, B. E., & Hubble, M. A. (2010). The heart and soul of change: Delivering what works in therapy (2nd ed.). Washington, DC: American Psychological Association.

Greenberg, L. S., Watson, J. C., Elliot, R., & Bohart, A. C. (2001). Empathy. Psychotherapy: Theory, research, practice, training, 38, 380–384. doi:10.1037/0033-3204.38.4.380

Gurman, A. S. (1977). The patient’s perception of the therapeutic relationship. In A. S. Gurman & A. M. Razin (Eds.), Effective psychotherapy: A handbook of research (pp. 503–543). New york, Ny: Pergamon Press.

Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61, 155–163. doi:10.1002/jclp.20108

Page 13: Monitoring Alliance and Outcome with Client Feedback Measures

Monitoring Alliance and Outcome

55

Hatcher, R. L., & Barends, A. W. (1996). Patients’ view of the alliance in psychotherapy: Exploratory factor analysis of three alliance measures. Journal of Consulting and Clinical Psychology, 64, 1326–1336. doi:10.1037/0022-006X.64.6.1326

Horvath, A. O., & Bedi, R. P. (2002). The alliance. In J. C. Norcross (Ed.), Psychotherapy relationships that work (pp. 37–69). New york, Ny: Oxford University Press.

Horvath, A. O., & Symonds, B. D. (1991). Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology, 38, 139–149. doi:10.1037/0022-0167.38.2.139

Howard, K. I., Kopta, S. M., Krause, M. S., & Orlinsky, D. E. (1986). The dose–effect relationship in psychotherapy. American Psychologist, 41, 159–164. doi:http://dx.doi.org/10.1037/0003-066X.41.2.159

Lambert, M. J. (2010). yes, it is time for clinicians to routinely monitor treatment outcome. In B. L. Duncan, S. D. Miller, B. E. Wampold, & M. A. Hubble (Eds.), The heart and soul of change: Delivering what works in therapy (2nd ed., pp. 239–266).Washington, DC: American Psychological Association.

Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68, 438–450. doi:10.1037/0022-006X.68.3.438

Miller, S. D., Duncan, B. L., Brown, J., Sparks, J. A., & Claud, D.A. (2003). The Outcome Rating Scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. Journal of Brief Therapy, 2, 91–100.

Miller, S. D., Duncan, B. L., & Hubble, M.A. (2004). Beyond integration: The triumph of outcome over process in clinical practice. Psychotherapy in Australia, 10, 2–19.

Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The Partners for Change Outcome Management System. Journal of Clinical Psychology, 61, 199–208. doi:10.1002/jclp.20111

Norcross, J. C. (2010). The therapeutic relationship. In B. L. Duncan, S. D. Miller, B. E.Wampold, & M. A. Hubble (Eds.), The heart and soul of change: Delivering what works in therapy (2nd ed., pp. 113–141).Washington, DC: American Psychological Association.

Orlinsky, D. E., Grawe, K., & Parks, B. K. (1994). Process and outcome in psychotherapy—Noch einmal. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (4th ed., pp. 270–378). New york, Ny: Wiley.

Orlinsky, D. E., Rønnestad, M. H., & Willutzki, U. (2004). Fifty years of process-outcome research: Continuity and change. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 307–390). New york, Ny: Wiley.

Raskin, N. J., Rogers, C. R., & Witty, M. C. (2008). Client-centered therapy. In R. J. Corsini & D. Wedding (Eds.), Current Psychotherapies (8th ed., pp. 141–186). Belmont, CA: Thomson Brooks/Cole.

Reese, R. J., Norsworthy, L. A., & Rowlands, S. R. (2009). Does a continuous feedback system improve psychotherapy outcome? Psychotherapy: Theory, research, practice, training, 46, 418–431. doi:10.1037/a0017901

Reese, R. J., Usher, E. L., Bowman, D. C., Norsworthy, L. A., Halstead, J. L., Rowlands, S. R., Chisholm, R. R. (2009). Using client feedback in psychotherapy training: An analysis of its influence on supervision and counselor self-efficacy. Training and Education in Professional Psychology, 3, 157–168. doi:10.1037/a0015673

Rogers, C. R. (1957). The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology, 21, 95–103. doi:10.1037/h0045357

Smith, G. R., Fischer, E. P., Nordquist, C. R., Mosley, C. L., & Ledbetter, N. S. (1997). Implementing outcomes management systems in mental health settings. Psychiatric Services, 48, 364–368. Retrieved from http://search.proquest.com/docview/619384106?accountid=14593

Stewart, R. E., & Chambless, D. L. (2007). Does psychotherapy research inform treatment decisions in private practice? Journal of Clinical Psychology, 63, 267–281. doi:10.1002/jclp.20347

Tyron, G. S., Blackwell, S. C., & Hammel, E. F. (2007). A meta-analytic examination of client-therapist perspectives of the working alliance. Psychotherapy Research, 17, 629–642.

Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Erlbaum.

Page 14: Monitoring Alliance and Outcome with Client Feedback Measures

56

Appendix A

Session Rating Scale (SRS V.3.0)

Name Age (Yrs):

ID# Sex: M / F

Session # Date:

Please rate today’s session by placing a hash mark on the line nearest to the description that best fits your experience.

Relationship:

I did not feel heard, understood, and respected.

I II felt heard, understood, and respected.

Goals and Topics:

We did not work on or talk about what I wanted to work on and talk about.

I I

We worked on and talked about what I wanted to work on and talk about.

Approach or Method:

The therapist’s approach is not a good fit for me.

I IThe therapist’s approach is a good fit for me.

Overall:

There was something missing in the session today.

I IOverall, today’s session was right for me.

International Center for Clinical Excellence

www.centerforclinicalexcellence.com

© 2002, Scott D. Miller, Barry L. Duncan, & Lynn Johnson

Note: Sample Copy. Working copies may be obtained in 25 different languages at http://scottdmiller.com/performance-metrics/

Page 15: Monitoring Alliance and Outcome with Client Feedback Measures

Monitoring Alliance and Outcome

57

Appendix B

Outcome Rating Scale (ORS)

Name Age (Yrs): Sex: M / F

Session # Date:

Who is filling out this form? Please check one: Self Other

If other, what is your relationship to this person?

Looking back over the last week (or since your last visit), including today, help us understand how you have been feeling by rating how well you have been doing in the following areas of your life, where marks to the left represent low levels and marks to the right indicate high levels. If you are filling out this form for another person, please fill out according to how you think he or she is doing.

Individually:(Personal well-being)

I I

Interpersonally:(Family, close relationships)

I I

Socially:(Work, school, friendships)

I I

Overall:(General sense of well-being)

I I

International Center for Clinical Excellence

www.centerforclinicalexcellence.com

© 2002, Scott D. Miller, Barry L. Duncan, & Lynn Johnson

Note: Sample Copy. Working copies may be obtained in 25 different languages at http://scottdmiller.com/performance-metrics/