www.elsevier.com/locate/bt

Available online at www.sciencedirect.com

Behavior Therapy 43 (2012) 468–481

What Makes a Quality Therapy? A Consideration of Parsimony,Ease, and Efficiency

Jesse R. CougleFlorida State University

Evaluations of psychotherapy have traditionally focused onsymptom reduction as the primary standard by which theirvalue is determined. This has contributed to the appearanceof equivalence between many therapies that may differconsiderably in complexity, feasibility, amount of home-work and therapist contact required, expected cost, speed ofsymptom decline, and transdiagnostic utility. In the currentpaper, I make the case that these are fundamental featuresrelated to quality that should be considered in psychother-apy development, randomized controlled trials, and dissem-ination efforts. Empirically supported treatments fordifferent disorders are evaluated based on these criteria,and special consideration is given to cognitive-behavioraltreatments for anxiety disorders. Specific recommendationsfor a quality-oriented clinical research agenda are alsoprovided.

Keywords: therapy efficiency; cognitive-behavioral therapy;exposure therapy; cognitive restructuring; treatment dissemination

12 Web site (www.PsychologicalTreatments.org)currently keeps an updated list of treatments with

VARIOUS FORCES IN and outside the scientificcommunity have worked to focus increasing atten-tion on identifying empirically supported interven-tions for psychiatric disorders.One of the products ofsuch efforts was the formation of the Task Force onPromotion and Dissemination of Psychological Pro-cedures of Division 12 (Clinical Psychology) of the

I would like to thank the anonymous reviewers for their helpfulcomments on an earlier version of this manuscript.

Address correspondence to Jesse R. Cougle, Ph.D., Departmentof Psychology, Florida State University, PO Box 3064301,Tallahassee, FL 32306; e-mail: cougle@psy.fsu.edu.0005-7894/43/468-481/$1.00/0© 2011 Association for Behavioral and Cognitive Therapies. Published byElsevier Ltd. All rights reserved.

American Psychological Association (Chambless,1995). This Task Force proposed several criteriafor identifying empirically supported treatments,including comparison of the treatment to anappropriate control group, the use of treatmentmanuals, and independent replication of therapyeffects. They also classified treatments according tothe number of such criteria they met, giving thedesignation of “well established” to those fulfillingall criteria and the “probably efficacious” title tothosemeeting fewer but not all criteria (Chambless etal., 1998). The Task Force originally identified 25treatments in 1995 (Chambless, 1995), though thisnumber grew to 71 in 1998 (Chambless et al., 1998).By 2001, the group identified 108well-established orprobably efficacious treatments for adults and 38 forchildren (Chambless&Ollendick, 2001). ADivision

modest or strong research support.Though the efforts of the Division 12 Task Force

are important and worthwhile, the high number ofempirically supported treatments identified is prob-lematic for a few reasons. First, the preponderanceof therapies makes it difficult for educators andpractitioners to identify which interventions toteach, learn, and utilize. It also raises issues ofquality control, as only those who are well trainedin specific approaches would be able to provideadequate supervision. Therapist competency maybe negatively impacted by the scarcity of expertsupervisors available, as well as the varied criteriafor proficiency likely to exist across differentschools of therapy. Further, and perhaps mostimportant, this list leaves the impression ofequivalence between therapies that may have keydifferences that would make one approach prefer-able to another.

1 These conclusions are of course based on the design of thisstudy. That is, the investigators did not examine the efficacy ofcognitive restructuring with and without prolonged exposure. Thefact that only certain components are usually targeted for removalis a limitation of dismantling studies.

469what make s a qual i ty therapy?

It would likely be asking too much for the TaskForce to further distinguish between treatmentswith strong empirical support. Besides beingfraught with controversy, such an endeavor wouldin many cases be going beyond the data. Indeed,insofar as they are primarily focused on treatmentefficacy, published clinical trials offer little guidancefor this problem. Many of the empirically sup-ported treatments listed do in general lead toequivalent outcomes.In the current essay, I hope to review as well as

present additional ways in which psychologicaltreatments might be evaluated, with the goal ofmoving beyond the criterion of symptom reduction.I propose that an overattentiveness to symptomreduction has served to distract the field fromimportant ways in which the quality of therapy canbe improved and empirically supported treatmentscan be distinguished. Instead, I argue for greaterattention to parsimony, ease, and efficiency inpsychotherapy. Different interventions will bediscussed in light of these criteria, with an emphasison cognitive-behavioral treatments for anxietydisorders. Some consideration will also be givento treatments for depression. In conclusion, Ipresent specific recommendations for a quality-oriented clinical research agenda.I should note that these proposed criteria for

psychotherapy evaluation have a significant degreeof overlap and have been mentioned by variousresearchers over the years. However, this is to myknowledge the first attempt to formally specifypotential features of a quality therapy in a singleessay. My goal in doing so is to spur additionalresearch and discussion.

The Importance of Parsimony in ClinicalInterventions

Given equivalent outcomes between two differenttherapies, the more parsimonious approach may bethe preferred approach. As I discuss below,parsimony in therapy, or the number and simplicityof therapeutic components, may have a directbearing on the amount of therapist trainingrequired to successfully implement therapy, theease of treatment dissemination, treatment integri-ty, and the degree to which clients adhere totreatment.

more parsimonious therapies mayrequire less clinical training andwill likely be easier to disseminate

Complex treatments will generally require a largeramount of clinical training than simpler ap-proaches. The less there is to teach, the less time,effort, and money will be required to learn.

Resource constraints may be less of a concern fordoctoral students undergoing years of clinicaltraining. However, full-time therapists, in particu-lar, will have difficulty leaving work and paying fordays or even weeks of clinical training andsupervision. Thus, the complexity of a treatmentwill likely influence how easily it can be dissemi-nated.A treatment with fewer components will not

necessarily require less training. Cognitive restruc-turing as a specific treatment component is argu-ably more complex and may require more trainingand supervision than relaxation, breathing retrain-ing, or exposure therapy. One study demonstratedsome level of success in teaching exposure therapyto nonclinicians using a 1-hour computerizedtraining course (Gega, Norman, & Marks, 2007).Unfortunately, there is scant research that speaks tothe relative difficulty of training clinicians towardadequate proficiency in different treatments.Foa and colleagues (2005a) conducted one

particularly relevant study in which individualswith PTSD were treated using prolonged exposurewith or without cognitive restructuring. Thetreatments were delivered by doctoral-level clini-cians from an academic research center andmaster's-level community therapists with littleprior experience with cognitive-behavioral therapy(CBT). All therapists were trained together in a 5-day workshop for prolonged exposure and asecond 5-day workshop for cognitive restructuring.No significant differences in outcome emergedbetween treatment conditions or between academicand community therapists. It is possible that theextra 5-day training in cognitive restructuring led tobenefits from prolonged exposure that would nothave otherwise appeared. However, these findingsstill suggest that prolonged exposure withoutcognitive restructuring would be the preferredtreatment for PTSD, as significantly more clinicaltraining was required for the cognitive restructuringcomponent that demonstrated no additive value.Though these therapies produced equivalent out-comes, the additional value conferred by greaterparsimony and less required training would tip thescale in favor of prolonged exposure withoutcognitive restructuring.1

Similarly, component analyses of cognitive ther-apy for depression have found that behavioralactivation is just as effective as the full treatment

470 cougle

package (Jacobson et al., 1996). Given the simplic-ity of this procedure and the extensive trainingrequired to learn the additional components ofcognitive therapy, behavioral activation wouldappear to be the preferred intervention for depres-sion. Meta-analytic findings also provide encour-aging evidence for the efficacy of behavioralactivation (Cuijpers, Van Straten, & Warmerdam,2007; Mazzucchelli, Kane & Rees, 2009).

greater parsimony may lead togreater treatment integrity

Among empirically supported interventions, moreparsimonious treatments may also be preferredbecause they are likely to lead to greater treatmentintegrity. Treatment integrity, or the degree towhich an intervention is administered as it wasintended (Yeaton & Sechrest, 1981), has beenimplicated as an essential factor in predictingtherapy success, though there is conflicting evidenceregarding its importance. A very recent meta-analysis found no overall relationship betweentreatment integrity and outcome (Webb, Derubeis,& Barber, 2010), though moderator analyses foundintegrity to predict outcome in studies of depres-sion. Some have argued that the absence ofassociations between integrity and outcome acrossdifferent studies may be due to limitations in theway integrity is usually assessed (Perepletchikova&Kazdin, 2005).More complex treatments with a greater number

of components may require more materials, re-sources, and sessions than simpler approaches.Given the limitations imposed by cost, time, clientattrition, and the number of procedures therapistswill be able to learn and implement effectively, thereshould be significant benefit associated with themost parsimonious treatment. Generally speaking,the lower the number and the greater the simplicityof treatment components, the greater the likelihoodthat each component will be used and the treatmentwill be implemented as intended.One effectiveness study of group treatment for

depression compared professionals to paraprofes-sionals who were trained to administer CBT or anondirective mutual support therapy (Bright,Baker, & Neimeyer, 1999). Overall, equivalentoutcomes were found between treatments and typesof therapists. However, among clients treated withCBT, those treated by professionals had higherrates of recovery than those treated by paraprofes-sionals. The authors speculated that the lowerefficacy shown by paraprofessionals administeringCBT may be due to the sophistication of techniquesrequired by this treatment. This study parallelsother clinical trials that have found greater therapist

experience to be associated with improved outcomein CBT for depression (DeRubeis et al., 2005;Jacobson & Hollon, 1996). Overall, this researchimplicates potential difficulties in implementingCBT that may not be present with simpler, lessstructured approaches.

greater parsimony may lead to greaterclient treatment adherence andimproved outcome

More complex treatments may lead to poorer clienttreatment adherence than simpler approaches.Some clients will have difficulty comprehendingvarious treatment components, and there will likelybe an inverse relationship between the number oftreatment components and overall treatment ad-herence. Thus, complex therapies may lead topoorer outcomes than simpler approaches.There has been very little research examining the

potential relationships between parsimony andclient treatment adherence and treatment outcome.Intelligence has not been found to predict outcomein individuals undergoing cognitive therapy(Haaga, DeRubeis, Stewart, & Beck, 1991; Rizvi,Vogt, & Resick, 2009); however, such studies maybe confounded by range restriction, behavioralcomponents of treatment, or skill level of clinicians.Illiterate populations, older adults, and individualswith intellectual disabilities are typically excludedfrom clinical trials of CBT. Procedures such asmonitoring of automatic thoughts, identification ofcognitive distortions, and cognitive reappraisal mayhave less utility for such individuals and in manycases will be impossible to implement. One studyevaluated the cognitive-emotional skills of individ-uals with mild intellectual disabilities and notedsome difficulties these individuals had in under-standing basic concepts (e.g., cognitive mediation)essential for cognitive therapy (Dagnan, Chadwick,& Proudlove, 2000).Cognitive therapies usually include certain com-

ponents that are simple enough for lower-function-ing clients, and the apparent benefits of cognitivetherapy for such individuals may be due to theeffects of these simpler components. For example,clients who may not comprehend or adhere well tothe full cognitive therapy package for depressionmay still benefit from the simpler behavioralactivation component that is used throughouttreatment. In his review of cognitive therapy forpeople with intellectual disabilities, Sturmey(2004) noted that it was difficult to ascertain theunique benefits of specific cognitive techniques forthis population because many interventions labeled“cognitive” consisted mostly of behavioral com-ponents. He concluded that more work is needed

471what make s a qual i ty therapy?

examining specific cognitive techniques to helpdetermine their efficacy with this group.Therapies for emotional disorders are usually

modified for use with children, older adults, andpopulations with cognitive impairment. For exam-ple, it has been recommended that cognitivetherapy for depressed older adults involve fewergoals and focus more on behavioral activation thancognitive restructuring (Crowther, Scogin, &Norton, 2010). Stripped-down or simplified adap-tations of therapies are also recommended forchildren (Friedberg & McClure, 2002) and forclients with intellectual disabilities (Lindsay, How-ells, & Pitcaithly, 1993). More parsimoniousinterventions may require fewer adaptations forthese populations.There are some data to suggest that greater

therapeutic complexity will negatively impacttherapy outcome. For example, McLean et al.(2001) found that behavior therapy (exposure andresponse prevention) was more effective than CBTwhen delivered in a group format for obsessive–compulsive disorder (OCD). The authors speculatedthat the group setting was not an optimal formatfor administering a treatment as complex as CBT,which in their study focused on six differentbelief domains. The increased efficiency and cost-effectiveness conferred by group behavior therapy nodoubt make it an especially attractive treatmentoption for OCD.Many interventions rely on knowledge and skills

that are accumulated over multiple therapy sessionsrather than specific techniques that can be quicklyimplemented. For example, cognitive therapies(e.g., Heimberg & Becker, 2002) typically involvepsychoeducation regarding thoughts and emotions,identification and self-monitoring of automaticthoughts, and identification of thinking errors, aswell as methods for challenging thinking errors.Sporadic or limited session attendance may reducethe likelihood that such procedures will have anyefficacy (Reardon, Cukrowicz, Reeves, & Joiner,2002).

The Importance of Ease inClinical Interventions

The quality of a therapy may also be influenced bythe ease with which it is implemented and thedifficulties it imposes on both therapist and client. Irefer to ease here as including feasibility, aversive-ness, number and duration of sessions, amount ofhomework, and cost. Each of these interrelatedcharacteristics has been neglected to varying degreesin psychotherapy evaluation, though I argue thatthey are all important in assessing the advantagesand disadvantages of different treatments.

feasibility of therapy

Psychotherapies that produce equivalent outcomesmay differ considerably in feasibility of implemen-tation. This issue is particularly relevant to anxietytreatments that rely on extensive exposure exer-cises. Lack of resources may make varioustreatment components difficult to use in typicalclinical settings. For example, recent CBT pro-tocols for social anxiety disorder utilize video andaudio feedback of client performance along withexposure exercises involving audiences and con-versational partners (Clark et al., 2006). Manyclinical settings do not have the equipmentnecessary for such exercises, and it is often difficultto coordinate in-session exposure exercises involv-ing an audience or conversational partner. Greaterdisclosure of the degree to which therapies rely onoutside resources would help assess the feasibilityof different treatment protocols. More specifically,anxiety researchers could report the averageamount and type of coordinated exposures perclient, which could include total audience orconversational partner exposures, as well as anyout-of-session exposures (e.g., driving, in-homeexposures). Generally speaking, interventions thatrequire less effort to implement successfully will bepreferred.Concerns of feasibility might make such methods

as role-playing and imaginal exposure more attrac-tive options for in-session use. Group therapysettings should make audience exposures mucheasier to conduct. In addition, different Internetsites may be helpful in facilitating exposure byproviding chat rooms for treating social anxiety orcombat-related audio recordings and video clips fortreating PTSD (e.g., http://www.ilovewavs.com/Effects/War/War.htm).

aversiveness of therapy

The emotional distress required by therapies shouldalso be considered when evaluating their quality. Isay this with caution, understanding that manydisorders are maintained by fear or avoidance ofanxiety and distress (Hayes, Wilson, Gifford,Follette, & Strosahl, 1996), and fear confrontationvia exposure therapy is an effective and relativelyparsimonious treatment approach. However, thereis a need for less aversive alternatives. Approxi-mately 25% of individuals with OCD refuseexposure and response prevention treatment(Franklin & Foa, 1998) and up to 28% drop outonce treatment begins (Foa, Liebowitz, Kozak et al.,2005b). Though reasons for exposure therapyrefusal have not been studied extensively, suchrefusal is no doubt due in part to apprehension

472 cougle

regarding the intensity and distress it requires.Similar hesitancies related to fear confrontation arelikely a major cause of the low rates of treatmentutilization among people with other anxiety disor-ders (Issakidis & Andrews, 2002).A number of alternatives exist that could benefit

those who refuse exposure therapy. CBT generallyincludes some elements of fear confrontation,though in the form of behavioral experimentsinvolving brief exposure rather than repeated,prolonged exposure (Clark et al., 1994). There issome evidence to suggest CBT is less aversive toclients than treatments involving repeated expo-sure. For example, research suggests that CBTresults in lower dropout rates than exposure andresponse prevention in the treatment of OCD(Abramowitz, Taylor, & McKay, 2005; Whittal,Robichaud, Thordarson, & McLean, 2008),though a meta-analysis of treatments for socialanxiety disorder found no differences in dropoutrates between behavior therapy and cognitivetherapy without exposure (Gould, Buckminster,Pollack, Otto, & Yap, 1997). Researchers surveyed852 psychologists and found that very few usedexposure for PTSD, and approximately halfthought the use of exposure would increase thelikelihood of client attrition (Becker, Zayfert, &Anderson, 2004). However, a review of publishedtreatment studies for PTSD found no differences inattrition rates between exposure therapy andcognitive therapy (Hembree et al., 2003).2

If cognitive techniques have benefits as lessaversive alternatives to exposure therapy, thesebenefits must also be balanced against other issuesof quality mentioned in this essay (e.g., parsimony,training requirements). Cognitive procedures mayalso require more time to implement effectively. Forexample, Öst, Alm, Brandberg, & Breitholtz (2001)found that one session of exposure was as effectiveas five sessions of cognitive restructuring (and fivesessions of exposure therapy) in the treatment ofclaustrophobia; unfortunately, they did not includea single-session cognitive restructuring conditionfor adequate comparison. Treatment protocolsincorporating as little as 30 min of exposure havedemonstrated response rates of up to 94% forclaustrophobia (Powers, Smits, & Telch, 2004); tomy knowledge, no studies have demonstrated

2 Comparing rates of refusal or attrition between differenttreatments is complicated by the fact that many clients are initiallytold that there is a likelihood that they will be assigned to atreatment involving fear confrontation. Many anxious clients alsoseek out treatment with the understanding that it will involvedistressing assessments and interventions. Thus, rates of refusal andattrition reported in clinical trials may underestimate problemsposed by distressing treatments.

comparable efficacy from cognitive restructuringin as short an amount of time. Additionally, a meta-analysis of treatments for social anxiety disorderfound that exposure therapy produced larger effectsizes than cognitive restructuring without exposure(Gould et al., 1997).Other noteworthy treatments that may be less

distressing than exposure therapy include appliedrelaxation (Öst, 1987), which has demonstratedefficacy in the treatment of various anxiety disor-ders. However, at least with regard to attrition, thisintervention appears to have no advantage overCBT (Siev & Chambless, 2007). Preliminaryevidence also suggests interpersonal psychotherapy,an additional intervention not requiring repeatedfear confrontation, is effective for the treatment ofPTSD (Bleiberg &Markowitz, 2005), though it hasnot been researched extensively.It is not yet clear whether cognitive therapy,

applied relaxation, and interpersonal psychothera-py are in fact less aversive than exposure therapy, atleast as measured by differences in refusal andattrition rates. To the degree that each of thesealternative treatments requires effort on the part ofthe client, they involve their own type of “discom-fort.” The acceptability and effectiveness of thesetherapies for clients who refuse exposure therapyhas thus far received little research attention. It ispossible that repeated exposure might be bestfacilitated by first having clients engage in relaxa-tion, cognitive restructuring, or other less distres-sing techniques, though the use of such techniquesmay require sacrifices in therapy efficiency. This iscertainly a topic deserving of further research.Further head-to-head comparisons of exposure-and non-exposure-based treatments would alsohelp identify individuals who are at risk of refusalor attrition who may be more likely to benefit fromeither type of treatment or perhaps a more gradualtreatment approach. The assessment of aversivenessthrough means other than refusal and attrition rateswould also be beneficial. This could be donethrough asking various questions related to clientperceptions of treatment, including satisfactionwith therapy, perceptions of the amount of distressor work required in therapy, and whether he or shewould use the treatment again or recommend it to afriend. Such basic questions are commonly asked inproduct surveys and would also be worth includingin randomized controlled trials.Furthermore, given that enrollment in random-

ized controlled trials often involves somewillingnesson the part of anxious individuals to confront theirfears (especially if there is some likelihood that theywould be assigned to exposure therapy), examina-tion of treatment perception among non-treatment-

473what make s a qual i ty therapy?

seeking individuals with anxiety disorders may bebeneficial in identifying more acceptable alterna-tives to exposure-based treatments. Studies that door do not allow anxious individuals to choose fromamong multiple psychotherapies, with varied ratio-nales and degrees of aversiveness, might also behelpful toward identifying factors that maximizetreatment utilization and minimize attrition.

treatment length

Treatment length is an important characteristicrelated to each feature of therapy quality I haveproposed and is discussed at several points in thisessay. Meta-analyses of brief interventions havegenerally found a dose relationship between num-ber of therapy sessions and greater likelihood ofsuccessful outcome (Howard, Kopta, Krause, &Orlinsky, 1986). However, due to the constraints ofcost, time, and effort, fewer sessions are generallypreferred. Hansen, Lambert, and Forman (2002)noted that clinical trials consisted of an average of12.7 sessions and produced improvements in 57.6–67.2% of participants; however, their analysis of anational database of more than 6,000 patientsrevealed that the average number of sessionsattended by this sample was less than five and theimprovement rate was approximately 20%. Giventhese real-world constraints, treatments with veryfew components that can be quickly and effectivelyimplemented should have great value.A balance between effort and effectiveness will be

an obvious consideration when attempting toestablish the optimal number of sessions. That is,if 20 additional sessions contributed to smalldecreases in symptoms or marginal increases inresponse rates, the inclusion of these extra sessionsin a standard therapy package is probably unwar-ranted. Cost–benefit analyses may lead researchersto develop multitiered interventions that vary inlength and efficacy. Perhaps longer, more effectiveinterventions could be reserved for certain settingsthat would allow for their full implementation.Some investigators have varied the number of

sessions and found equivalent outcomes betweenfull and condensed treatments. For example, re-searchers have found a condensed 6-session cogni-tive therapy to be just as effective as the full 12-session treatment for panic disorder (Clark et al.,1999; Kenardy et al., 2003). This research isimportant to consider when conducting newrandomized controlled trials, since it would suggestthat this 6-session CBT package is the preferredtreatment to which newer therapies should becompared. Single-session interventions have alsoshown impressive efficacy for both anxiety anddepression (Gawrysiak, Cristopher, & Hopko,

2009; Kunik et al., 2001; Öst et al., 2001).Generally speaking, treatments that require moresessions should be able to show improved quality insome regard (e.g., greater efficacy, feasibility).The literature on sudden gains might be helpful in

determining the optimal number of sessions for aparticular client (Tang & DeRubeis, 1999). Clientswho experience early, marked improvements mayrequire fewer sessions to achieve lasting recovery.Research that examines predictors of sudden gainsor other factors related to chronicity and remissionmay be informative in developing a treatment planthat requires the least number of sessions. Rando-mized controlled trials could also be conducted thatcompare fixed-session frequency to session frequen-cy that is varied based on the experience of sud-den gains or pretreatment predictors of remission/chronicity.Session length also varies between treatments,

generally lasting between 1 and 2 hours. Again,given the constraints of time, cost, and effort,shorter sessions will be preferred. A meta-analysisby Abramowitz (1996) found that greater length oftherapist-assisted exposure was associated withimproved outcome in exposure and response pre-vention treatment for OCD. In contrast, vanMinnenand Foa (2006) recently found that PTSD treatmentusing 30 min of imaginal exposure in each ses-sion was just as effective as treatment using 60 minof exposure per session. Similar studies comparingshorter and longer therapy sessions would go a longway toward improving therapy quality. The burdenwill be on investigators using longer sessions todemonstrate that their treatments have additive valueover those with shorter sessions.

amount of required homework

Despite the fact that homework is quite integral tomany psychotherapies, there has been little effort toquantify the amount of homework required byvarious treatments and assess the effort suchhomework requires. Available evidence suggestsboth the quantity and quality of homeworkcompletion is predictive of treatment outcome(Kazantzis, Deane, & Ronan, 2000; Mausbach,Moore, Roesch, Cardenas, & Patterson, 2010).While Schmidt and Woolaway-Bickel (2000) foundclinician-rated quality and percentage of homeworkassignments completed were associated with treat-ment outcome for CBT for panic disorder, clientreports of number of hours spent on homework wasnot associated with outcome. Other researchersexamining homework compliance and behaviortherapy outcome for individuals with OCD orpanic disorder with agoraphobia found no rela-tionship between therapy outcome and quantity or

474 cougle

quality of homework (Woods, Chambless, &Steketee, 2002). Interestingly, these researchersnoted that their clients spent an average of46 hours on homework throughout the course oftreatment. The absence of an association betweenhomework and outcome may be due to unreliabilityof client self-report. Regardless, these findingssuggest that the amount of assigned homeworkcould be scaled back without harming treatmentefficacy. Clinical trials that experimentally vary theamount of homework could do much to enhancetherapy quality. Generally speaking, if two differenttreatments produce equivalent outcomes, the treat-ment requiring less and easier homework ispreferred.3 Reducing the amount of assignedhomework may also reduce the risk of attrition.

cost of therapy

The expected cost of therapy is an obviousconsideration when evaluating its quality, andmany features of therapy discussed in this essayhave cost implications. Duration and number oftreatment sessions are important contributors. Theformat in which therapy is delivered is also a factor,with group treatments costing less to clients thanindividual treatment (Otto, Pollack, & Maki,2000). Computer-aided treatments (Craske et al.,2009) will have great advantages in terms of cost-effectiveness. Relatively parsimonious therapiesthat can be effectively delivered by paraprofes-sionals with little clinical training will also help re-duce therapy expense (and increase dissemination),as paraprofessionals will likely charge less for ther-apy than doctoral-level psychologists (Christensen& Jacobson, 1994). The administration of psycho-education and other treatment components throughself-study modules can also help minimize therapistcontact and reduce cost; this was primarily howClark et al. (1999) condensed the aforementioned12-session treatment for panic disorder to 6 ses-sions. In short, there are many opportunities forreducing the cost of psychotherapy.

The Importance of Efficiency inClinical Interventions

One of the hallmarks of a quality therapy should beits efficiency. In general, highly efficient therapieswill not include ineffective strategies and will seekto maximize the speed of symptom decline. Themost efficient interventions should also demon-strate transdiagnostic utility.

3On a related point, I could not identify any publishedrandomized controlled trial that compared the amount of home-work required and completed between two different empiricallysupported treatments.

speed of symptom reduction

A quality therapy will seek to maximize the speed atwhich symptoms are reduced. The importance ofrapid symptom reduction lies in the impairment andcomorbidity associated with the symptoms them-selves, the possibility of treatment dropout (result-ing in a lower likelihood of ever benefiting fromtreatment), the low number of sessions attended bymany clients, and of course the wishes of clientsto reduce symptoms as quickly as possible. Theexpected speed of symptom reduction is no doubtone reason why many individuals with psychiatricdisorders choose medication over psychotherapy,and there is some evidence to suggest medicationsreduce symptoms more rapidly (Heimberg et al.,1998).Speed of symptom reduction should be an

obvious consideration for treatment researchers,though it is often neglected in outcome research.Most studies assess and report symptoms only atpre- and posttreatment. More frequent assessmentswould help determine differences in speed of effectsbetween therapies as well as the optimal numberof recommended treatment sessions. Componentanalyses would also help identify ineffective orless effective treatment procedures that could bereplaced with faster, more efficient methods.Research assessing rates and predictors of suddengains for particular treatments (Tang & DeRubeis,1999) would also be relevant toward this aim.

transdiagnostic utility

Psychiatric disorders rarely occur alone (Brown,Campbell, Lehman, Grisham, & Mancill, 2001);thus, the quality of a therapy may be determined inpart by its transdiagnostic utility. Such utility can bedemonstrated in multiple ways, including the impactof an intervention on comorbid conditions, the easewith which components from an intervention can beutilized or adapted for treating multiple disorders,andwhether an intervention comes froma theoreticalperspective broad enough to account for multipledisorders (cf. Barlow, Allen, & Choate, 2004).These features of therapy have several implica-

tions for efficiency. First, the degree to whichtherapy impacts maintaining factors occurringacross disorders may affect overall treatmentoutcome, as well as the total number of sessionsrequired. Second, therapists and clients will be ableto transition more easily between specific interven-tions for different disorders if these procedures arevery similar or come from similar theoreticalperspectives. This might allow them to consolidateand build on therapeutic gains. Third, a protocolthat can be used for treating multiple disorders

475what make s a qual i ty therapy?

may require less overall training and less time spenton assessment and case formulation for multipledisorders. It might also allow for diagnosticheterogeneity in a group therapy setting.There is now an abundance of empirical support

for CBT for various disorders (Norton & Price,2007). The explanatory power of cognitive theoryis no doubt a compelling reason to use CBT.However, a potential weakness of CBT is the rise ofso many disorder-specific dysfunctional beliefs. Forexample, recent cognitive-behavioral approaches toOCD target beliefs related to inflated responsibility,perfectionism, intolerance for uncertainty, over-importance of thoughts, control of thoughts, andoverestimation of threat (McLean et al., 2001).Maladaptive beliefs about memory have also beennoted (Cougle, Salkovskis, & Wahl, 2007). ForPTSD, dysfunctional beliefs related to self-blame,dangerousness of the world, and emotional re-actions to the trauma have been proposed (Foa,Ehlers, Clark, Tolin, & Orsillo, 1999), and cog-nitive processing therapy in particular focuses ondysfunctional beliefs related to safety, trust, powerand control, self-esteem, self-intimacy, and intima-cy with others (Resick & Schnicke, 1993). Target-ing so many different disorder-specific beliefs maynegatively impact therapy efficiency. In general, thelower the number of targeted maintaining factors,the more efficiently they may be addressed. I shouldnote here that one of the attractive features ofacceptance and commitment therapy (ACT; Hayeset al., 1996) is its emphasis on experientialavoidance as a key maintaining factor occurringacross psychiatric disorders.Transdiagnostic limitations can be found in other

therapies, as well. For example, interpersonalpsychotherapy has established efficacy for thetreatment of depression (Frank & Spanier, 1995),but there is less support for its use with OCD orpanic disorder; thus, it may be less efficient to usethis treatment for depressed clients with certaincomorbid profiles. In addition, despite the fact thatprolonged exposure and eye movement desensiti-zation and reprocessing (EMDR; Shapiro, 1995)are equivalent in efficacy for PTSD (Seidler &Wagner, 2006), the degree to which prolongedexposure complements and overlaps with exposure-based treatments for other anxiety disorders makesit preferable to EMDR, which represents a moredistinct trauma-oriented approach.4

Recently, researchers have created treatment pro-tocols for use with multiple disorders. A transdiag-

4 EMDR has also been heavily criticized by many for includingcomponents with no incremental value (Herbert, Lilienfeld, & Lohret al., 2000).

nostic treatment for emotional disorders has beendeveloped that focuses on emotional awareness, cog-nitive restructuring, emotional and behavioral avoid-ance, and exposure (Ellard, Fairholme, Boisseau,Farchione, & Barlow, 2010). Group therapy pro-tocols have also been produced for the treatmentof anxiety disorders (see this issue for Norton, 2012-this issue; and Schmidt, Buckner, Pusser,Woolaway-Bickel, & Preston, 2012-this issue). Whether suchinterventions represent true advances in therapeuticefficiency remains to be seen. Empirical evaluationsof these interventionswill hopefully examine (a) howthey compare to existing disorder-specific treatmentsin terms of efficacy and required training, (b) theimpact of such interventions on comorbidity,(c) whether such treatments would be more usefulor efficient than existing disorder-specific treatmentsimplemented with an equivalent degree of flexibility,and (d) what procedures in these interventions areessential for producing symptom reduction.

Quality-Oriented Research RecommendationsI have hopefully covered many different issuesrelated to therapy quality that are worthy of furtherresearch. In Table 1, I provide a summary ofresearch recommendations. I have devoted exten-sive discussion to parsimony in this essay, but thereis currently very little research that has tested itspotential significance. One way in which its impor-tance can be evaluated is by examining the relativeease and effectiveness with which different treat-ments and treatment components can be taughtto and implemented by community clinicians andparaprofessionals. This would be extremely rele-vant to treatment dissemination efforts. The typesof randomized controlled training studies con-ducted by Strosahl, Hayes, Bergan, and Romano(1998), who assigned therapists to receive ACT-based training or a control condition, would beespecially beneficial. Head-to-head comparisons oftraining in different therapies, along with compre-hensive assessments of client outcomes and thera-pist proficiencies, are needed to evaluate many ofthe features of therapy mentioned in this essay.Analysis of the decay of proficiency and treatmentcomponent utilization over time would also beimportant to incorporate into effectiveness studies.In addition, “mini-effectiveness studies” in whichundergraduate research assistants, hospital person-nel, or other easily accessible samples are randomlyassigned to training in different approaches to treatspider fearful, blood phobic, or claustrophobicindividuals in single-session interventions wouldaddress some of these research issues. Effortstoward increasing parsimony will also be aided bydismantling studies and component analyses. If

Table 1Research Recommendations to Improve the Quality of Psychotherapies

Issues Related to Qualityof Therapy

Potential Means to Improve

Increasing Parsimony- Compare the effectiveness of treatments that vary in complexity and number of components

- Examine the amount of training required to implement different treatments effectively

- Assess client comprehension of various components across different populations

- Test therapies using newly trained clinicians and paraprofessionals

- Conduct dismantling studies and component analyses; use findings from such studies to revise protocol

- Prioritize techniques that can be implemented quickly and do not rely on knowledge or skillsaccumulated over multiple sessions

Increasing EaseIncreasing feasibility - Develop and utilize interventions that are easy to implement in a standard community clinic setting

- Administer treatment through computers or other accessible formats (e.g., group, informal class)

- Identify highly accessible stimuli and situations for exposure therapyReducing aversiveness - Examine or develop less distressing alternatives to exposure therapy and test their efficacy for

individuals who refuse exposure-based treatment

- Develop instruments that fully assess client perception of different treatments and their aversivenessReducing treatment length - Test variations in the number of sessions and session length to help determine optimal quantities for

each

- Assess symptoms frequently to determine the optimal point at which therapy can be discontinued

- Examine predictors of sudden gains or early remission to identify whether therapy might be terminatedearlier for certain clients

Reducing the amount ofrequired homework

- Quantify and compare the amount of homework assigned and completed for different treatments

- Test variations in the amount of assigned homework to help determine the most optimal quantityReducing cost - Develop treatments that can be administered in a group format

- Use computers and other technologies to aid in the implementation of therapy

- Develop treatments simple enough to be administered by paraprofessionals

- Use take-home readings to replace therapy time spent on psychoeducation and other procedures

Increasing EfficiencyIncreasing speed of symptomreduction

- Use frequent symptom assessments and component analyses to assess the relative speed with whichdifferent treatments and treatment components reduce symptoms

- If possible, frontload components in therapy that are most likely to have the quickest effects onsymptoms

Increasing transdiagnosticutility

- Develop interventions targeting factors thought to maintain multiple disorders

- Develop interventions that can be utilized or easily adapted for the treatment of multiple disorders

- Utilize interventions that build on or complement existing treatments targeting other disorders

- Compare the impact of different interventions on target problems and comorbid conditions

476 cougle

ineffective or nonessential components are identi-fied through such studies, their removal couldimprove therapy protocols. Researchers are likelyto see benefits from making treatments withestablished efficacy simpler.With regard to ease, it will be important for

researchers to develop interventions that are highlyfeasible for implementing in a traditional commu-nity clinic setting. The extent to which procedureswith questionable feasibility (e.g., the use ofaudience exposures) are used in a therapy shouldbe reported by treatment researchers, as this willaffect the generalizability of treatment outcome

findings. Further efforts should be made to quantifythe optimal number and length of sessions, as wellas the optimal amount of homework required foreffective treatment. Generally speaking, researchersshould attempt to produce highly effective treat-ments requiring less, easier homework and fewer,shorter sessions. Research examining sudden gainsor predictors of remission/chronicity may beinformative in determining the optimal sessionlength for a particular client. In addition, as longas such efforts do little or no damage to treatmentefficacy, researchers should seek to reduce thediscomfort associated with therapy or develop

Tab

le2

AHyp

othe

tical

Com

paris

onof

theQua

lityof

Three

Anx

iety

Disorde

rTreatmen

ts

Parsimon

yEas

eEfficien

cyEffica

cy

Num

berof

Com

pone

nts

Leve

lof

Sop

histication

Amou

ntof

TrainingReq

uired

toIm

plem

ent

Effe

ctively

Amou

ntof

Assigne

dExp

osure

Amou

ntof

Ass

igne

dHom

ework

Attrition

Rate

Evide

ncefor

Cos

t-Effe

ctive

Delivery

Num

berof

Ses

sion

Hrs

Until50

%Sym

ptom

Red

uctio

n/Total

Ses

sion

Hrs

Ove

rlapWith

Treatmen

tsforOther

Disorde

rs

Rec

overy

Rate

Leve

lof

Res

earch

Sup

port

(Div.12

)

Treatmen

tA

4Mod

erate

30hrs

15hrs

30hrs

25.4%

Para-profes

sion

als

compu

ter-as

sisted

grou

p

7.3/12

Som

ewha

t70

.5%

Stron

g

Treatmen

tB

2Lo

w10

hrs

20hrs

25hrs

27.2%

Para-profes

sion

als

grou

p5.8/12

High

65.3%

Stron

g

Treatmen

tC

6High

40hrs

12hrs

35hrs

24.2%

Non

e6.8/15

Mod

erate

63.8%

Mod

est

477what make s a qual i ty therapy?

alternative interventions that allow clients togradually work up to aversive procedures. Moredetailed assessments of perceptions of varioustreatment components in both treatment-seekingand non-treatment-seeking samples would helpestablish whether certain procedures may besubstituted for more distressing ones. Direct com-parisons of therapies that vary in levels ofaversiveness may also assist researchers in identify-ing clients who are likely to benefit more from lessdistressing treatments.Therapy efficiency concerns should also lead

researchers to seek to maximize the speed at whichsymptoms are reduced. Component analyses andrandomized controlled trials utilizing frequentassessments (to compare rates of symptom declinebetween treatments) would aid such efforts. Lastly,researchers should develop treatments with hightransdiagnostic value. Ideally, such interventionswould impact maintaining factors appearing acrossdisorders and comorbid conditions that are notnecessarily the focus of treatment. They would alsoinclude components that could be easily utilized oradapted for treating multiple disorders.In Table 2, I provide a hypothetical example of a

quality-oriented comparison of three anxiety disor-der treatments. Some indices, including sophistica-tion and amount of overlap with treatments forother disorders, carry with them a greater degreeof subjective evaluation than others, though per-haps more objective rating systems could be de-veloped that better evaluate treatments on thesecharacteristics.

Should Cognitive Restructuring Be Part of aQuality Therapy?

There has been growing debate over the value ofcognitive techniques in therapy (Longmore &Worrell, 2007). While most if not all behavioralinterventions include some cognitive procedures(e.g., discussion of thoughts, feared consequences),evidence has accumulated that suggests the additionof more focused cognitive procedures does notconfer any benefit over behavioral activation fordepression (Jacobson et al., 1996) and exposure-based interventions for PTSD (Foa et al., 2005a),OCD (Whittal et al., 2008), and social anxietydisorder (Feske & Chambless, 1995). While oneunpublished study found that cognitive restructur-ing plus exposure led to more favorable outcomesthan exposure alone for the treatment of panicdisorder (Margraf & Schneider, 1991), publishedresearch has demonstrated equivalent outcomesbetween CBT and interoceptive exposure (withoutin vivo exposure) for this disorder (Arntz, 2002). Arecent meta-analysis also found applied relaxation,

478 cougle

arguably a more parsimonious treatment, to be justas effective as CBT for generalized anxiety disorder(Siev & Chambless, 2007).What is noteworthy about much of the research

demonstrating no incremental value from cognitivetechniques is that it has been carried out mostly byprominent research teams with allegiances to CBT.They typically used very skilled clinicians who arehighly proficient in CBT. If additive value fromcognitive procedures was not demonstrated bythese investigators, it is much less likely to befound among less proficient community clinicianswho undergo less training and have less supervi-sion. To my knowledge, there is no publishedresearch that has found CBT to be more effectivethan behaviorally oriented interventions whenimplemented by newly trained community clini-cians.Behavioral activation and exposure-based in-

terventions have great advantages over existingcognitive-behavioral treatments in terms of parsi-mony. They are likely easier for clients to under-stand and for therapists to learn and utilize. Theycan be quickly implemented and do not typicallyrely on knowledge and skills accumulated overmultiple sessions. One study also found behavioralactivation to be more effective than CBT for severedepression, though it produced similar outcomes toCBT for less severe depression (Dimidjian et al.,2006). In addition, exposure and response preven-tion appears to be more effective than CBT forOCD when conducted in group format (McLeanet al., 2001). Very brief exposure therapy protocolshave shown remarkable effectiveness (e.g., Powerset al., 2004), and it is possible that exposure-basedinterventions lead to more rapid fear reduction thancognitive restructuring; to date, this has receivedlittle research attention. While some acknowledgethat the addition of cognitive techniques toexposure-based interventions does not yield betteroutcomes, they maintain that such techniques areless aversive and have utility for this reason(Heimberg & Ritter, 2008). However, evidencefor their substitutionary value is underwhelming,with lower attrition rates revealed for CBT thanbehavior therapy for OCD (Whittal et al., 2008)but comparable rates of attrition between cognitiveinterventions and behavior therapy for socialanxiety disorder (Gould et al., 1997) and PTSD(Hembree et al., 2003). Given the rarity with whichtherapists in the general population use empiricallysupported treatments (e.g., Becker et al., 2004), itwould behoove researchers to attempt to developand disseminate effective interventions that aremost easily learned and implemented. Addingnonessential, labor intensive cognitive procedures

to simple, effective interventions may hamperefforts to ensure that therapists use empiricallysupported treatments.Given these treatment outcome findings and the

issues involved with administering more complexinterventions, one might ask whether we as a fieldshould be disseminating cognitive procedures.However, it would be important here to distinguishbetween different procedures. Behavioral experi-ments, for example, can be implemented morequickly and efficiently than cognitive restructuring,and there is some evidence to suggest that exposurein the context of a behavioral experiment (i.e.,identifying threats and evaluating them throughexposure) is more effective than exposure only(McMillan & Lee, 2010). A recent clinical trialfound CBT that relied on several behavioralexperiments led to better outcomes in the treatmentof social anxiety disorder than exposure therapywith applied relaxation (Clark et al., 2006). Inaddition, metacognitive therapy, which uses acombination of verbal strategies and behavioralexperiments, appears to be very effective forgeneralized anxiety disorder (Wells et al., 2010), acondition notoriously difficult to treat. It is possiblethat cognitive restructuring has unique benefits forthis (highly cognitive) disorder. On the whole,however, there are many reasons for the field tocarefully evaluate whether cognitive restructuringspecifically should be included as a treatmentcomponent for most psychological disorders.

Balancing Efficacy and Various Featuresof Therapy Quality

Several features of therapy quality mentioned in thisessay, such as number and length of sessions,aversiveness, and cost, may at times conflict witheach other and with goals of symptom reduction.For example, it is possible that less aversiveinterventions may reduce symptoms at a slowerspeed than interventions that begin with exposure.Various forms of cost-effective interventions (e.g.,treatments requiring fewer sessions, treatmentsdelivered via computer or by paraprofessionals)may also be less effective than more expensiveinterventions. Different features may be prioritizedon a case-by-case basis or based on the needsassociated with a particular practice. Less aversivetreatments that may also be less effective or slowerin producing symptom change may be warrantedfor a client who refuses exposure therapy. Treat-ment enrollment and client retention should gener-ally be prioritized over the need for immediatesymptom reduction. Shorter, higher-impact inter-ventions may be warranted if a client is likely toattend few sessions or show sporadic session

479what make s a qual i ty therapy?

attendance. Constraints of session attendanceshould inform the approach used to maximizesymptom reduction. From a dissemination stand-point, these concerns may warrant training thera-pists in different techniques that would beadaptable for use with clients at risk of refusal orattrition and those likely to attend relatively fewsessions. Of course, such approaches should beempirically informed. Additional research that teststherapies that vary in levels of aversiveness andduration would help address these issues.

Beyond Symptom ReductionOver the last several years, psychotherapy researchhas focused excessively on pre- to posttreatmentreduction in symptoms as the primary criterionby which the quality of therapy is determined. Thisemphasis is in some ways equivalent to a carmanufacturer being only concerned with whethertheir product reliably delivers the driver from pointA to point B, while neglecting its safety, comfort,speed, and cost. The efficacy of treatments for manydisorders seems to have peaked 20 to 30 years ago(e.g., Whittal et al., 2008). Such stagnancy istroubling to many researchers. However, if greatereffort is made toward examining the additionalfeatures of therapy discussed in this essay, thiscould lead to new and meaningful improvements inthe quality of our product.

ReferencesAbramowitz, J. S. (1996). Variants of exposure and response

prevention in the treatment of obsessive-compulsive disorder:A meta-analysis. Behavior Therapy, 27, 583–600.

Abramowitz, J. S., Taylor, S., & McKay, D. (2005). Potentialsand limitations of cognitive treatments for obsessive-compulsive disorder. Cognitive Behaviour Therapy, 34,140–147.

Arntz, A. (2002). Cognitive therapy versus interoceptiveexposure as treatment of panic disorder without agoraphobia.Behaviour Research and Therapy, 40, 325–341.

Barlow, D. H., Allen, L. B., & Choate, M. L. (2004). Toward aunified treatment for emotional disorders. Behavior Therapy,35, 205–230.

Becker, C. B., Zayfert, C., & Anderson, E. (2004). A survey ofpsychologists’ attitudes towards and utilization of exposuretherapy for PTSD. Behaviour Research and Therapy, 42,277–292.

Bleiberg, K. L., & Markowitz, J. C. (2005). A pilot study ofinterpersonal psychotherapy for posttraumatic stress disorder.American Journal of Psychiatry, 162, 181–183.

Bright, J. I., Baker, K. D., & Neimeyer, R. A. (1999).Professional and paraprofessional group treatments fordepression: A comparison of cognitive-behavioral andmutual support interventions. Journal of Consulting andClinical Psychology, 67, 491–501.

Brown, T. A., Campbell, L. A., Lehman, C. L., Grisham, G. R.,& Mancill, R. B. (2001). Current and lifetime comorbidityof the DSM-IV anxiety and mood disorders in a large clinicalsample. Journal of Abnormal Psychology, 110, 585–599.

Chambless, D. (1995). Training in and dissemination ofempirically-validated psychological treatments: Report andrecommendations. Clinical Psychologist, 48, 3–23.

Chambless, D., Baker, M. J., Baucom, D. H., Beutler, L. E.,Calhoun, K. S., Crits-Christoph, P., et al. (1998). Update onempirically validated therapies: II. Clinical Psychologist, 49,5–18.

Chambless, D., & Ollendick, T. (2001). Empirically supportedpsychological interventions: Controversies and evidence.Annual Review of Psychology, 52, 685–716.

Christensen, A., & Jacobson, N. S. (1994). Who (or what) cando psychotherapy: The status and challenge of nonprofes-sional therapies. Psychological Science, 5, 8–14.

Clark, D. M., Ehlers, A., Hackmann, A., McManus, F., Fennell,M., Grey, N., et al. (2006). Cognitive therapy versusexposure and applied relaxation in social phobia: Arandomized controlled trial. Journal of Consulting andClinical Psychology, 74, 568–578.

Clark, D. M., Salkovskis, P. M., Hackmann, A., Middleton, H.,Anastasiades, P., & Gelder, M. (1994). Comparison ofcognitive therapy, applied relaxation and imipramine in thetreatment of panic disorder. British Journal of Psychiatry,164, 759–769.

Clark, D. M., Salkovskis, P. M., Hackmann, A., Wells, A.,Ludgate, J., & Gelder, M. (1999). Brief cognitive therapy forpanic disorder: A randomised controlled trial. Journal ofConsulting and Clinical Psychology, 67, 583–589.

Cougle, J. R., Salkovskis, P.M.,&Wahl, K. (2007). Perception ofmemory ability and confidence in recollections in obsessive-compulsive checking. Journal of Anxiety Disorders, 21,118–130.

Craske, M. G., Rose, R. D., Lang, A., Welch, S. S., Campbell-Sills, L., Sullivan, G., et al. (2009). Computer-assisteddelivery of cognitive behavioral therapy for anxiety disor-ders in primary-care settings. Depression and Anxiety, 26,235–242.

Crowther, M. R., Scogin, F., & Norton, M. J. (2010). Treatingthe aged in rural communities: The application of cognitive-behavioral therapy for depression. Journal of ClinicalPsychology, 66, 502–512.

Cuijpers, P., Van Straten, A., & Warmerdam, L. (2007).Behavioral activation treatments of depression: A meta-analysis. Clinical Psychology Review, 27, 318–326.

Dagnan, D., Chadwick, P., & Proudlove, J. (2000). Towardassessment of suitability of people with mental retardationfor cognitive therapy. Cognitive Therapy and Research, 24,627–636.

DeRubeis, R. J., Hollon, S. D., Amsterdam, J. D., Shelton, R. C.,Young, P.R., Salomon,R.M., et al. (2005). Cognitive therapyvs. medications in the treatment of moderate to severedepression. Archives of General Psychiatry, 62, 409–416.

Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B.,Kohlenberg, R. J., Addis, M. E., et al. (2006). Randomizedtrial of behavioral activation, cognitive therapy, andantidepressant medication in the acute treatment of adultswith major depression. Journal of Consulting and ClinicalPsychology, 74, 658–670.

Ellard, K. K., Fairholme, C. P., Boisseau, C. L., Farchione, T. J.,& Barlow, D. H. (2010). Unified protocol for thetransdiagnostic treatment of emotional disorders: Protocoldevelopment and initial outcome data. Cognitive andBehavioral Practice, 17, 88–101.

Feske, U., & Chambless, D. L. (1995). Cognitive behavioralversus exposure only treatment for social phobia: A meta-analysis. Behavior Therapy, 26, 695–720.

Foa, E. B., Ehlers, A., Clark, D.M., Tolin, D. F.,&Orsillo, S.M.(1999). The Posttraumatic Cognitions Inventory (PTCI):

480 cougle

Development and validation. Psychological Assessment, 11,303–314.

Foa, E. B., Hembree, E. A., Cahill, S. P., Rauch, S. A. M., Riggs,D. S., Feeny, N. C., et al. (2005a). Randomized trial ofprolonged exposure for posttraumatic stress disorder withand without cognitive restructuring: Outcome at academicand community clinics. Journal of Consulting and ClinicalPsychology, 73, 953–964.

Foa, E. B., Liebowitz, M. R., Kozak, M. J., Davies, S., Campeas,R., Franklin, M. E., Huppert, J. D., et al. (2005b). Treatmentof obsessive-compulsive disorder by exposure and ritualprevention, clomipramine, and their combination: A ran-domized, placebo controlled trial. American Journal ofPsychiatry, 162, 151–161.

Frank, E., & Spanier, C. (1995). Interpersonal psychotherapy fordepression: Overview, clinical efficacy, and future directions.Clinical Psychology: Science and Practice, 2, 349–369.

Franklin, M., & Foa, E. B. (1998). Cognitive-behavioraltreatments for obsessive compulsive disorder. In P. N. A. J.Gorman (Ed.), A guide to treatments that work(pp. 339–357). New York: Oxford University Press.

Friedberg, R. D., & McClure, J. M. (2002). Clinical practice ofcognitive therapy with children and adolescents: The nutsand bolts. New York: Guilford Press.

Gawrysiak, M., Cristopher, N., & Hopko, D. R. (2009).Behavioral activation for moderately depressed universitystudents: Randomized controlled trial. Journal of Counsel-ing Psychology, 56, 468–475.

Gega, L., Norman, I., & Marks, I. (2007). Computer-aided vs.tutor-delivered teaching of exposure therapy for phobia/panic:A randomised controlled trial with pre-registration nursingstudents. International Journal of Nursing Studies, 44,397–405.

Gould, R. A., Buckminster, S., Pollack, M. H., Otto, M. W., &Yap, L. (1997). Cognitive-behavioral and pharmacologicaltreatment for social phobia: A meta-analysis. ClinicalPsychology: Science and Practice, 4, 291–306.

Haaga, D. A. F., DeRubeis, R. J., Stewart, B. L., & Beck, A. T.(1991). Relationship of intelligence with cognitive therapyoutcome. Behaviour Research and Therapy, 29, 277–281.

Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002). Thepsychotherapy dose-response effect and its implications fortreatment delivery services. Clinical Psychology: Science andPractice, 9, 329–343.

Hayes, S. C., Wilson, K. G., Gifford, E. V., Follette, V. M., &Strosahl, K. (1996). Experiential avoidance and behavioraldisorders: A functional dimensional approach to diagnosisand treatment. Journal of Consulting and Clinical Psychol-ogy, 64, 1152–1168.

Heimberg, R. G., & Becker, R. E. (2002). Cognitive-behavioralgroup therapy for social phobia: Basic mechanisms andclinical strategies. New York: Guilford Press.

Heimberg, R. G., Liebowitz,M. R., Hope, D. A., Schneier, F. R.,Holt, C. S., Welkowitz, L. A., et al. (1998). Cognitivebehavioral group therapy vs. phenelzine therapy for socialphobia: 12-week outcome. Archives of General Psychiatry,55, 1133–1141.

Heimberg, R. G., & Ritter, M. R. (2008). Cognitive behavioraltherapy and acceptance and commitment therapy for theanxiety disorders: Two approaches with much to offer.Clinical Psychology: Science and Practice, 15, 296–298.

Hembree, E. A., Foa, E. B., Dorfan, N. M., Street, G. P.,Kowalski, J., & Tu, X. (2003). Do patients drop outprematurely from exposure therapy for PTSD? Journal ofTraumatic Stress, 16, 555–562.

Herbert, J. D., Lilienfeld, S. O., Lohr, J.M., Montgomery, R.W.,O'Donohue, W. T., Rosen, G. M., et al. (2000). Science and

pseudoscience in the development of eye movementdesensitization and reprocessing: Implications for clinicalpsychology. Clinical Psychology Review, 20, 945–971.

Howard, K. I., Kopta, S. M., Krause, M. K., & Orlinsky, D. E.(1986). The dose-effect relationship in psychotherapy.American Psychologist, 41, 159–164.

Issakidis, C., & Andrews, G. (2002). Service utilisation foranxiety in an Australian community sample. Social Psychi-atry and Psychiatric Empidemiology, 37, 153–163.

Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E.,Koerner, K., Gollan, J. K., et al. (1996). A component analysisof cognitive-behavioral treatment for depression. Journal ofConsulting and Clinical Psychology, 64, 295–304.

Jacobson, N. S., & Hollon, S. D. (1996). Prospects forfuture comparisons between drugs and psychotherapy:Lessons from the CBT-versus-pharmacotherapy exchange.Journal of Consulting and Clinical Psychology, 64,104–108.

Kazantzis, N., Deane, F. P., & Ronan, K. R. (2000). Homeworkassignments in cognitive and behavioral therapy: A meta-analysis. Clinical Psychology: Science and Practice, 7,189–202.

Kenardy, J. A., Dow, M. G. T., Johnston, D. W., Newman, M.G., Thomson, A., & Taylor, C. B. (2003). A comparison ofdelivery methods of cognitive-behavioral therapy for panicdisorder: An international multicenter trial. Journal ofConsulting and Clinical Psychology, 71, 1068–1075.

Kunik, M. E., Braun, U., Stanley, M. A., Wristers, K., Molinari,V., Stoebner, D., & Orengo, C. A. (2001). One sessioncognitive behavioral therapy for elderly patients with chronicobstructive pulmonary disease. Psychological Medicine, 31,717–723.

Lindsay, W. R., Howells, L., & Pitcaithly, D. (1993). Cognitivetherapy for depression with individuals with intellectualdisabilities. British Journal of Medical Psychology, 66,135–141.

Longmore, R. J., & Worrell, M. (2007). Do we need tochallenge thoughts in cognitive behavior therapy? ClinicalPsychology Review, 27, 173–187.

Margraf, J., & Schneider, S. (1991, November 26). Outcomeand active ingredients of cognitive-behavioural treatmentsfor panic disorder. Paper presented at the Annual Conferenceof Association for Advancement of Behaviour Therapy,New York, NY.

Mausbach, B. T., Moore, R., Roesch, S., Cardenas, V., &Patterson, T. L. (2010). The relationship between home-work compliance and therapy outcomes: An updatedmeta-analysis. Cognitive Therapy and Research, 34(5),429–438.

Mazzucchelli, T., Kane, R., & Rees, C. (2009). Behavioralactivation treatments for depression in adults: A meta-analysis and review. Clinical Psychology: Science andPractice, 16, 383–411.

McLean, P. D., Whittal, M. L., Thordarson, D., Taylor, S.,Sochting, I., Koch, W. J., et al. (2001). Cognitive versusbehavior therapy in the group treatment of obsessive–compulsive disorder. Journal of Consulting and ClinicalPsychology, 69, 205–214.

McMillan, D., & Lee, R. (2010). A systematic review ofbehavioral experiments vs. exposure alone in the treatmentof anxiety disorders: A case of exposure while wearing theemperor's new clothes? Clinical Psychology Review, 30,467–478.

Norton, P. J. (2012). A randomized clinical trial of transdiag-nostic cognitve-behavioral treatments for anxiety disorderby comparison to relaxation training. Behavior Therapy,43, 506–517 (this issue).

481what make s a qual i ty therapy?

Norton, P. J., & Price, E. C. (2007). A meta-analytic review ofadult cognitive-behavioral treatment outcome across theanxiety disorders. Journal of Nervous and Mental Disease,195, 521–531.

Öst, L. G. (1987). Applied relaxation: Description of a copingtechnique and review of controlled studies. BehaviourResearch and Therapy, 25, 379–409.

Öst, L. G., Alm, T., Brandberg, M., & Breitholtz, E. (2001).One vs. five sessions of exposure and five sessions ofcognitive therapy in the treatment of claustrophobia.Behaviour Research and Therapy, 39, 167–183.

Otto, M. W., Pollack, M. H., & Maki, K. M. (2000).Empirically supported treatments for panic disorder:Costs, benefits, and stepped care. Journal of Consultingand Clinical Psychology, 68, 556–563.

Perepletchikova, R., & Kazdin, A. E. (2005). Treatmentintegrity and therapeutic change: Issues and researchrecommendations. Clinical Psychology: Science and Practice,12, 365–383.

Powers, M. B., Smits, J. A. J., & Telch, M. J. (2004).Disentangling the effects of safety-behavior utilization andsafety-behavior availability during exposure-based treatment:A placebo-controlled trial. Journal of Consulting and ClinicalPsychology, 72, 448–454.

Reardon, M. L., Cukrowicz, K. C., Reeves, M. D., & JoinerJr.,T. E. (2002). Duration and regularity of therapy attendanceas predictors of treatment outcome in an adult outpatientpopulation. Psychotherapy Research, 12, 273–285.

Resick, P. A., & Schnicke, M. K. (1993). Cognitive processingtherapy for rape victims. Newbury Park, CA: Sage.

Rizvi, S. L., Vogt, D. S., & Resick, P. A. (2009). Cognitive andaffective predictors of treatment outcome in cognitiveprocessing therapy and prolonged exposure for posttrau-matic stress disorder. Behaviour Research and Therapy, 47,737–743.

Schmidt, N. B., Buckner, J. D., Pusser, A., Woolaway-Bickel, K.,& Preston, J. L. (2012). Randomized controlled trial of falsesafety behavior elimination therapy: A unified cognitivebehavioral treatment for anxiety psychopathology. BehaviorTherapy, 43, 518–532 (this issue).

Schmidt, N. B., & Woolaway-Bickel, K. (2000). The effectsof treatment compliance on outcome in cognitive-behavioral therapy for panic disorder: Quality versusquantity. Journal of Consulting and Clinical Psychology,68, 13–18.

Seidler, G. H., & Wagner, F. E. (2006). Comparing the efficacyof EMDR and trauma focused cognitive behaviouraltherapy in the treatment of PTSD: A meta-analytic study.Psychological Medicine, 36, 1515–1522.

Shapiro, F. (1995).Eyemovement desensitization and reprocessing:Basic principles, protocols andprocedures.NewYork:GuilfordPress.

Siev, J., & Chambless, D. L. (2007). Specificity of treatmenteffects: Cognitive therapy and relaxation for generalizedanxiety and panic disorders. Journal of Consulting andClinical Psychology, 75, 513–522.

Strosahl, K. D., Hayes, S. C., Bergan, J., & Romano, P. (1998).Does field based training in behavior therapy improveclinical effectiveness? Evidence from the acceptance andcommitment therapy training project. Behavior Therapy,29, 35–64.

Sturmey, P. (2004). Cognitive therapy with people withintellectual disabilities: A selective review and critique.Clinical Psychology and Psychotherapy, 11, 223–232.

Tang, T., & DeRubeis, R. J. (1999). Sudden gains and criticalsessions in cognitive-behavioral therapy for depression.Journal of Consulting and Clinical Psychology, 67,894–904.

Van Minnen, A., & Foa, E. B. (2006). The effect of imaginalexposure length on outcome of treatment for PTSD. Journalof Traumatic Stress, 19, 427–438.

Webb, C. A., Derubeis, R. J., & Barber, J. P. (2010). Therapistadherence/competence and treatment outcome: A meta-analytic review. Journal ofConsulting andClinical Psychology,78, 200–211.

Wells, A., Welford, M., King, P., Papageorgiou, C., Wisely, J.,& Mendel, E. (2010). A pilot randomized trial ofmetacognitive therapy vs applied relaxation in the treatmentof adults with generalized anxiety disorder. BehaviourResearch and Therapy, 48, 429–434.

Whittal, M., Robichaud, M., Thordarson, D., & McLean, P.(2008). Group and individual treatment of obsessive-compulsive disorder using cognitive therapy and exposureplus response prevention: A 2-year follow-up of tworandomized trials. Journal of Consulting and ClinicalPsychology, 76, 1003–1014.

Woods, C. M., Chambless, D. L., & Steketee, G. (2002).Homework compliance and behavior therapy outcome forpanic with agoraphobia and obsessive compulsive disorder.Cognitive Behaviour Therapy, 31, 88–95.

Yeaton, W., & Sechrest, L. (1981). Critical dimensions in thechoice and maintenance of successful treatments: Strength,integrity, and effectiveness. Journal of Consulting andClinical Psychology, 49, 156–167.

RECEIVED: April 23, 2010ACCEPTED: December 14, 2010Available online 25 May 2011