This article was downloaded by: [Memorial University of Newfoundland]On: 14 July 2014, At: 09:52Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Cognitive Behaviour TherapyPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/sbeh20

Does Experience Matter? TraineeExperience and Outcomes DuringTransdiagnostic Cognitive-BehavioralGroup Therapy for AnxietyPeter J. Nortona, Tannah E. Littlea & Chad T. Wetterneckb

a Department of Psychology, University of Houston, 126 HeyneBuilding, Houston, TX 77204-5022, USAb Rogers Memorial Hospital, 34700 Valley Road, Oconomowoc,WI 53066, USAPublished online: 04 Jun 2014.

To cite this article: Peter J. Norton, Tannah E. Little & Chad T. Wetterneck (2014) DoesExperience Matter? Trainee Experience and Outcomes During Transdiagnostic Cognitive-Behavioral Group Therapy for Anxiety, Cognitive Behaviour Therapy, 43:3, 230-238, DOI:10.1080/16506073.2014.919014

To link to this article: http://dx.doi.org/10.1080/16506073.2014.919014

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms

& Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

Does Experience Matter? Trainee Experience andOutcomes During Transdiagnostic Cognitive-

Behavioral Group Therapy for Anxiety

Peter J. Norton1, Tannah E. Little1* and Chad T. Wetterneck2**1Department of Psychology, University of Houston, 126 Heyne Building, Houston, TX 77204-5022, USA; 2Rogers Memorial Hospital, 34700 Valley Road, Oconomowoc, WI 53066, USA

Abstract. Research examining the impact of therapist experience on client outcomes has yieldedmixed findings. The current study aimed to improve upon previous research by examining the impactof global trainee therapists’ experience, as well as treatment protocol-specific experience, on clientoutcomes. Data were obtained based on 319 clients being treated by 33 therapists using a 12-weektransdiagnostic cognitive-behavioral group therapy specifically for anxiety disorders. Resultsdemonstrated that clients overall showed significant improvement in self-reported anxiety andclinician severity ratings, and that the amount of therapist experience was unrelated to improvement.The current study suggests that trainee therapists’ experience, whether examined as global amount oftherapy experience or specific experience delivering a specific treatment protocol, was unrelated totreatment outcomes or treatment discontinuation across a range of outcomes. Key words: anxiety;cognitive behavior therapy; experience; training.

Received 26 March 2014; Accepted 24 April 2014

Correspondence address: Peter J. Norton, Ph.D., Department of Psychology, University of Houston,126 Heyne Building, Houston, TX 77204-5022, USA. Tel: 713-743-8675. Fax: 713-743-8633. Email:pnorton@uh.edu

Anxiety disorders are psychological diagnosesthat involve maladaptive anxiety or worryabout a number of life domains or objects(e.g., spiders, social performance, physiologi-cal symptoms, contamination, traumaticevents, or daily life stressors) and causeclinically significant distress and/or impair-ment in social, occupational, and/or otherareas of functioning (American PsychiatricAssociation, 2013). Cognitive-behavioraltherapy (CBT) and exposure therapy havedemonstrated efficacy for all anxiety disordersin individual and group therapy settings, withonly minimal differences in outcomes acrossdiagnoses (Norton, 2012a; Norton & Price,2007). However, studies have shown that, atleast roughly, 20–50% of patients drop out orotherwise do not benefit from treatment(Norton & Hope, 2005). Thus, it is imperativeto investigate factors which may contributeto improved outcomes. One such factor thathas received some empirical attention is theimpact of the degree of therapist training

and experience, generally and specifically withanxiety disorder treatments, on outcomes.Many master’s and doctoral level therapists

receive training in their respective graduateprograms, in which clients frequently presentfor low cost services in exchange for seeingtherapists-in-training. However, there is apaucity of research exploring the effects oftherapist experience, education, and specializ-ation on treatment outcomes for mentaldisorders, particularly for anxiety disordersspecifically, and extant research has shownmixed or inconclusive results (Driscoll et al.,2003; van Oppen et al., 2010). For instance,some studies have demonstrated no significantdifferences in client outcomes betweenmaster’s-or doctoral-level trainees and certified pro-fessional therapists, suggesting that therapyadministered by trainees is just as efficacious astherapy administered by certified professionals(Forand, Evans, Haglin, & Fishman, 2011;Nyman, Nafziger, & Smith, 2010; Schmidt,Buckner, Pusser, Woolaway-Bickel, & Preston,

q 2014 Swedish Association for Behaviour Therapy

Cognitive Behaviour Therapy, 2014

Vol. 43, No. 3, 230–238, http://dx.doi.org/10.1080/16506073.2014.919014

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

2012; van Oppen et al., 2010). In contrast, Swiftand Greenberg (2012) provided evidencesuggesting higher treatment dropout rateswhen providers were trainees rather than post-degree professionals. Moreover, studies haveillustrated that client contact hours of trainees ispositively related to improved outcomes,indicating that greater experience is associatedwith better outcomes (Driscoll et al., 2003;Powell, Hunter, Beasley, & Vernberg, 2010).However, these studies assessed outcomesacross a variety of presenting problems anddisorders, as well as theoretical orientations.

Only three studies have investigated therelationship between therapist experience andclient outcomes for the treatment of anxietydisorders. van Oppen et al. (2010) comparedoutcomes during CBT for obsessive–compul-sive disorder (OCD) between experienced,certified behavior therapists and master’sstudents of clinical psychology and found nosignificant differences in outcomes. However,this study did not take into account thetherapists’ accrued experience (e.g., years orhours working with clients) or type ofexperience (e.g., experience in treating specifictypes of disorders and therapy orientations oftreatments provided). In addition, Huppertet al. (2001) examined the effects of therapistexperience, both conducting therapy generallyand using CBT specifically, on treatmentoutcomes for panic disorder patients.In contrast to van Oppen et al. (2010), Huppertet al. findings indicated that the amount oftherapist experience in conducting therapy waspositively associated with outcome on twosecondary measures of panic severity, but noton the primary or other secondary outcomes,and that experience with CBT was not relatedto outcome. Finally, Schmidt et al. (2012)reported no differences in outcomes during atransdiagnostic anxiety treatment studybetween masters-level therapists (1–2 years ofexperience) and a postdoctoral-level therapist.1

There are methodological problems inprevious studies examining the extent towhich therapist experience affects treatmentoutcomes for mental disorders in general.For instance, most studies do not controlfor the type of therapist/trainee experience(i.e., experience with treating specific problemsor disorders) or therapist training/experiencein specific theoretical treatment orientations.Also, discrepancies exist across studies in how

treatment outcomes are operationalized.For example, some studies assess change frompre- to post-treatment in outcome measures(e.g., Driscoll et al., 2003; Schmidt et al., 2012;van Oppen et al., 2010), while other studiesexamine dropout rates (e.g., Powell et al., 2010).

Furthermore, whether or not trainees usemanualized treatment protocols may also affectoutcomes. While the use of manualized treat-ment protocols is predictive of improvement foranxiety and depression (Crits-Christoph et al.,2001), few studies have investigated the efficacyof using manualized treatments among thera-pist trainees. Two studies have suggested thatthere are no differences in treatment outcomesacross levels of trainee experience when usingmanualizedCBTprotocols (Forand et al., 2011;Schmidt et al., 2012; van Oppen et al., 2010).In contrast, one study in which manualizedprotocols were used found that greater traineeexperience was modestly related to improvedoutcomes (Driscoll et al., 2003). However,these conflicting findings may be due to theaforementioned methodological issues anddiscrepancies.

Of note, Driscoll et al. (2003) examined therelationship of trainee therapy experience onoutcomes in a university-based, outpatient,evidence-based treatment clinic. Contrary tomuch of the previous research, Driscoll et al.(2003) found that number of client contacthours accrued by termination of treatmentwas positively associated with greater patientimprovement, although this only accountedfor 4% of the variability in improvement.While the results of Driscoll et al. (2003) arehighly informative, there are limitations totheir study. First, they included a samplecomposed of a variety of diagnoses. Second,all therapy hours were considered equivalent,that is, the therapy experience of two clinicianswho were both treating an individual withpanic disorder would be considered equivalenteven if one therapist had all 20 hours of theirexperience treating patients with panic dis-order versus the other therapist having all oftheir 20 hours of experience treating patientswith major depression.

Moreover, type of therapy experience maybe relevant because it may impact treatmentoutcomes. For instance, research has demon-strated that anxiety patients with therapistswho specialize in CBT for anxiety disordershave better outcomes and are less likely to

VOL 43, NO 3, 2014 Trainee Experience and Outcomes 231

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

relapse two years after treatment thananxiety patients with non-specialist therapists(Howard, 1999). Therefore, it is important toaccount for type of trainee experience whenassessing the effects of trainee experience onpatient outcomes.Thus, the purpose of the current study was

to examine the impact of (a) global therapistexperience and (b) treatment protocol-specificexperience on outcomes following a 12-weektransdiagnostic group CBT for anxiety dis-orders (see Norton, 2008; 2012a; Norton &Barrera, 2012). The current study aimed toimprove upon previous research by using amanualized protocol, assessing the amount oftime providing supervised psychotherapy, andassessing individual experience in deliveringthe specific treatment protocol. As previousresearch findings have been mixed, and thecurrent study is the first, to our knowledge, toexplore the impact of therapist experience ontreatment outcomes for group therapy, specifi-cally for anxiety disorders, we had littlebasis for generating substantial hypothesesalthough the majority of previous researchappears to suggest that there is little impact oftrainee experience on patient outcomes.

Method

ParticipantsClients. Client data were obtained from 319individuals attending services at the Universityof Houston Anxiety Disorder Clinic. Partici-pants were recruited for participation viaadvertisements and articles in local and neigh-borhood newspapers, referrals from health andmental health professions, and public servicemedia announcements. The following criteriawere established for inclusion in the study:(a) age 18 or older, (b) principal Diagnostic andStatistical Manual of Mental Disorders-IV(DSM-IV) diagnosis of any anxiety disorder,(c) proficiency in English, (d) no evidence ofdementia or other neurocognitive conditions,and (e) absence of serious suicidality or otherconditions that would require immediateintervention.The sample of treatment initiators consisted

of 151 men and 168 women, and was raciallydiverse (55.5% Caucasian, 22.6% Hispanic/Latino, 10.3%African-American, 6.9%AsianAmerican, 4.4% Other or Mixed racial back-ground, and 0.3% Native American). The

sample ranged in age from 18 to 71 years old(mean ¼ 33.00, (SD ¼ 10.52). Most weresingle (52.7%) or married (30.7%), and werewell educated (32.0% some undergraduatecourse, 30.1% bachelor’s degree or equivalent,20.7% professional/graduate school).Participants were assigned to treatment

groups based on order of presentation to theclinic. No efforts were made to influence thecomposition of the group by diagnosis orother characteristic. In all, clients from61 groups (mean n per group ¼ 5.23 clients,SD ¼ 1.71, range ¼ 2–9) were included.Initial group enrollments may have beenhigher due to clients being assigned to agroup but not attending any sessions.

Measures

All participants received a structured diagnos-tic assessment at intake and post-treatment,the Anxiety Disorders Interview Schedulefor DSM-IV (ADIS-IV; Brown, Di Nardo, &Barlow, 1994), as well as Clinician SeverityRatings (CSR) for each diagnosis, and anoverall Clinical Global Impressions (CGI)severity score. Participants also completedone self-report measure, the State-TraitAnxiety Inventory—State Version (STAI-S;Spielberger, 1983) immediately before thebeginning of each treatment session.Anxiety Disorders Interview Schedule forDSM-IV. The ADIS-IV (Brown, Di Nardo,& Barlow, 1994) is a semi-structured diag-nostic interview designed to assess thepresence, nature, and severity of DSM-IVanxiety, mood, and somatoform disorders, aswell as previous mental health history. Theinterview also contains a brief screen forpsychotic symptoms, and alcohol or substanceabuse. All ADIS-IV interviewers, advanceddoctoral students, were trained to reliabilitystandards by observing an interview con-ducted by an experienced interviewer thenconducting at least three interviews underobservation. A reliable match involved match-ing the experienced interviewer on diagnosesand matching the (CSR; see below) within 1point for the primary diagnosis. A large scaleanalysis of the ADIS-IV offers strong supportfor the reliability of diagnoses using the ADIS-IV (Brown, Di Nardo, Lehman, & Campbell,2001). The ADIS-IV assessment was adminis-tered to clients at both pre-treatment and

232 Norton, Little and Wetterneck COGNITIVE BEHAVIOUR THERAPY

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

post-treatment. Post-treatment ADIS-IVassessments were administered by indepen-dent evaluators who were blind to clientprogress during the course of group treatment.A random subset of ADIS-IV interviewswere rated by a second blind interviewer, andinter-rater diagnostic agreement was found tobe high (86% agreement, k ¼ .759, p , . 001).Clinician Severity Ratings. CSR, a componentof theADIS-IV, are subjective ratings applied bydiagnosticians to quantify the degree of severityfor each disorder diagnoses with the ADIS-IV.CSR range from 0 (not at all severe) to 8(extremely severe/distressing). Post-treatmentclinician ratings were assessed by independentevaluators who were blind to group therapyprogress, although the timing of the assessment(pre- vs. post-treatment) was unable to beblinded. Diagnostic reliability assessors showedstrong consistency in CSR ratings (Intra-ClassCorrelation (ICC) ¼ .774) with originalassessors.Clinical Global Impressions. Diagnosticiansalso completed the CGI Severity scale(National Institute of Mental Health, 1985),a clinician-rated measure of overall severity.As with the CSR ratings, CGI severity ratingswere made blind to group therapy progress,although not to the timing of the assessment.State-Trait Anxiety Inventory—State Version.The STAI-S (Spielberger, Gorsuch, Luschene,Vagg, & Jacobs, 1993) is a 20-item measuredesigned to assess state anxiety. The psycho-metric properties of the STAI-S are strong acrossmultiple populations (Spielberger et al., 1993),with anxiety disorder sample means rangingfrom 44 to 61 (see Antony, Orsillo, & Roemer,2001), and the measure has shown sensitivity totreatment effects (e.g., Fisher & Durham, 1999).At the initial session, the STAI-S was highlyinternally consistent in this sample (a ¼ .95).

ProcedureAssessment and treatment were conducted atthe University of Houston Anxiety DisorderClinic. All methods and procedures werereviewed by the Institutional Review Boardof University of Houston. Potential partici-pants were scheduled for the structureddiagnostic evaluation. Following the evalu-ation, participants eligible for participationwere enrolled in a cognitive behavioraltransdiagnostic group for anxiety. Informedconsent was obtained from all participants.

Treatment protocol and therapists. Therapistswere 33 graduate students (85.3% female) whoco-led at least one transdiagnostic CBTgroup. Therapists ranged in age from 22 to36 and were predominantly Caucasian(82.4%), although African-American (8.8%),Hispanic (5.9%), and Asian (2.9%) therapistswere represented. All therapists were studentsin a 4-year (plus pre-doctoral clinical intern-ship) scientist-practitioner Ph.D. program inClinical Psychology. In order to co-lead agroup, therapists were not required to have aminimum number of face-to-face clinicalhours. However, all therapists were requiredto have at least some basic level of coursetraining, including two courses in each ofpsychopathology, assessment, and clinicalinterventions. Therapist experience was com-puted in two ways: (1) the number of years(including fractions thereof) of graduate-leveltraining in clinical psychology, includingprior MA-level training if applicable,prior to the initiation of each transdiagnostictreatment group (hereafter termed “yearsof trainee therapy experience”) and (2) thenumber of previous transdiagnostic CBTgroups co-led by the therapist prior to eachnew group.

Treatment consisted of 12 weekly 2-hoursessions following a manualized protocol(Norton, 2012b).Therapists were supervisedby the first author. All therapists were trainedin the treatment protocol and were then pairedwith senior graduate student co-therapistswho had previously delivered the treatment,and both therapists were equally responsiblefor the delivery of each session. The seniorauthor directly observed all sessions forsupervision purposes and to ensure treatmentfidelity. None of the study authors conductedany ADIS interviews or treatment sessions inthe data reported here. Fidelity ratings weremade by trained research assistants who wereblind to ongoing treatment outcomes using anunpublished fidelity rating instrument devel-oped in concert with the treatment manual(available from the first author upon request).Ratings of a subset of sessions indicated thattherapists’ delivery of treatment was highlyconsistent with the protocol (Mean: 4.77/5.00,SD ¼ 0.24, range: 4.17–5.00). Primary out-comes (reported in Norton, 2008; 2012a;Norton & Barrera, 2012) suggested consider-able average improvement throughout treat-

VOL 43, NO 3, 2014 Trainee Experience and Outcomes 233

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

ment, although significant variability in out-comes across participants remained.

Analytic planThe impact of therapist experience on out-comes was examined as (a) the combinednumber of years of experience between the co-therapists and (b) the combined number ofprevious transdiagnostic CBT groups co-ledby each of the therapists.The data were modeled with a maximum

likelihood (ML) estimator in MPlus 4.1(Muthen & Muthen, 2006) using ML ANCO-VAs for measures assessed at pre- and post-treatment, or ML Mixed Effect RegressionModeling for the STAI, which was assessed atthe beginning of each treatment session. TheML estimator is advantageous over OrdinaryLeast Squares ANOVA models in thatmissing data can be effectively estimated,assuming the data are missing completely atrandom (Little & Rubin, 2002). Multipletime-point data were nested within individ-uals, and individuals were nested within theirrespective treatment group to account forindividual- and group-level effects. The datawere coded to reflect cases with (yes; 59.2%)or without (no; 40.8%) any missing data, andthe two groups did not differ on pre-treatmentCSR (F ¼ 1.58, p ¼ .21) or CGI ratings(F ¼ 0.42, p ¼ .84). Furthermore, slopes andsession 12 intercept scores were estimatedacross sessions using available STAI scores ina mixed-effect regression model, and bothslopes and session 12 intercepts were found tobe invariant across those with and withoutmissing data (sample-size adjusted BayesianInformation Criterion (BIC) [freelyestimated]) ¼ 13,464.79; sample-size adjustedBIC [invariant] ¼ 13,460.19). Power analysessuggested that the sample was sufficientlypowered (1 2 b ¼ .80, a ¼ .05) to detectassociations as small as r ¼ .16, or approxi-mately 2% shared variability.

Results

Preliminary client analysesOf the sample, 142 (44.5%) received a primarydiagnosis of social anxiety disorder, 88 (27.5%)panic disorder with or without agoraphobia, 58(18.2%) generalized anxiety disorder (GAD),11 (3.4%) OCD, 12 (3.7%) anxiety disorderNot Otherwise Specified (NOS) 7 (2.2%)

specific phobia, and one (0.3%) post-traumaticstress disorder (PTSD). Over half (62.4%) ofthe sample was given one or more additionaldiagnoses, based on lower CSR scores, includ-ing major depressive disorder, dysthymia, orother depressive mood disorder (n ¼ 102),GAD (n ¼ 70), social anxiety disorder(n ¼ 48), specific phobia (n ¼ 24), panic dis-order with or without agoraphobia (n ¼ 22),substance abuse or dependence (n ¼ 16), PTSD(n ¼ 6), and other diagnoses (all ns , 2).Clients attended an average of 7.46 sessions(SD ¼ 4.05), with a median of 8.00 and themodal number of sessions being 10. Consistentwith discontinuation rates from university-based outpatient clinics (Swift & Greenberg,2012), roughly one-third (34.5%) of the samplewere considered treatment dropouts in thatthey attended less than 75% of the sessions.

Preliminary therapist analysesSenior therapists were identified as thetherapist with the most experience deliveringthe treatment protocol. In cases in whichboth therapists had equal experience withthe treatment protocol, the therapist with thegreater overall number of years of therapyexperience was designated the senior therapist.At the initiation of each group, the seniortherapist had an average of 2.89 years(SD ¼ 1.08, range ¼ 1.12–5.81) of graduatetraining in clinical psychology, while the juniortherapist had an average of 2.19 years(SD ¼ 1.00, range ¼ 0.42–4.52). Summed, thegroup leaders had an average of 5.09 years(SD ¼ 1.47, range ¼ 2.07–8.74) of graduatetraining in clinical psychology. Similarly, seniortherapists had co-led an average of 3.55(SD ¼ 2.03) prior transdiagnostic CBT groups,while the junior therapist had co-led an averageof 0.81 (SD ¼ 1.21) previous transdiagnosticCBT groups. Summed, the group leaders hadled an average of 4.36 (SD ¼ 2.55) groups.Therapist years of clinical experience andnumber of previous groups co-lead weremoderately correlated, r ¼ .45, p , .001.

Effect of therapist years of clinicalexperience on outcomesClinician-rated measures. To examine therelationship between therapist years of clinicaltraining (summed across co-therapists) andclinical outcomes, pre- and post-treatment

234 Norton, Little and Wetterneck COGNITIVE BEHAVIOUR THERAPY

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

measures were entered into a within-groupsML ANCOVA where the post-treatmentmeasure was set as the intercept to estimateend-of-treatment functioning. The therapistexperience variable was then entered as apredictor of both treatment slope (rate ofimprovement) and intercept (end-point func-tioning). Both clinician rated measures (CSR ofthe primary diagnosis and overall CGI) wereseparately modeled and both showed signifi-cantly decreasing slopes (CSR slope ¼ 22.41,p ¼ .02; CGI slope ¼ 21.70, p ¼ .05) andsubclinical mean post-treatment intercepts(CSR icept ¼ 3.18; CGI icept ¼ 2.93). Inneither model did years of clinical experiencesignificantly predict slope (CSR p ¼ .79;CGI p ¼ .93) or end-of-treatment intercept(CSR p ¼ .80; CGI p ¼ .94).Self-report measures. Similarly, when model-ing STAI scores across each of the 12sessions, scores showed significant decreases(slope ¼ 0.98, p , .001) and an end-of-treat-ment intercept of 36.88. Total number of yearsof therapist clinical experience was not signifi-cantly related to either STAI slope (p ¼ .80) orend-of-treatment intercept (p ¼ .44).

Effect of past therapist experience withthe treatment protocol on outcomesTo examine the possible impact of therapistexperience with the specific treatment proto-col, the previous analyses were repeated,substituting the number of previously co-ledtransdiagnostic CBT groups as the index oftherapist experience.

Clinician-rated and self-report measures.A number of previous transdiagnostic groupswere entered as predictors of both treatmentslope (rate of improvement) and intercept(end-point functioning) for each of the CSRand CGI slope and intercept values. In neithermodel did number of previous transdiagnosticCBT groups co-led significantly predict slope(CSR p ¼ .77; CGI p ¼ .78) or end-of-treat-ment intercept (CSR p ¼ .85; CGI p ¼ .90).Similarly, when modeling STAI scores acrosssessions, scores showed significant decreases(slope ¼ 0.98, p , .001) and an end-of-treat-ment intercept of 36.88. Total number ofprevious transdiagnostic CBT groups co-ledwas not significantly related to either STAIslope ( p ¼ .80) or end-of-treatment intercept( p ¼ .44).

Effect of therapist experience ontreatment discontinuationThe impact of therapist experience on treat-ment dropout (failure to attend at least 75% ofthe treatment sessions) was examined usinga series of logistic regression analyses. Thefirst logistic regression used total years oftherapist training as the predictor variable, andno significant relationship was observed,OR ¼ 1.11, 95% CI ¼ 0.95–1.29, p ¼ .21.The second, using total number of transdiag-nostic CBT groups previously co-led, alsoshowed no relationship to dropout, OR ¼ 1.01,95% CI ¼ 0.92–1.10, p ¼ .79.

Individual effect of either senior orjunior therapist experience on outcomesor dropoutTo examine the possibility that either thesenior therapist’s experience would compen-sate for any possible junior therapist deficits,or the competing possibility that groupswould be constrained by the junior therapist’slesser experience, all analyses were recom-puted using just the “years of experience” or“previous transdiagnostic CBT groups co-led”variables from each of the junior and seniortherapist separately.Senior therapist effects. Examination of theeffect of senior therapist experience showedno significant relationship with outcomes asmeasured by the clinician-rated CGI (slope:p ¼ .85, icept: p ¼ .86) or CSR (slope: p ¼ .87,icept: p ¼ .89), self-reported STAI (slope:p ¼ .68, icept: p ¼ .98), or treatment discon-tinuation (OR ¼ 1.05, 95% CI ¼ 0.85–1.30,p ¼ .64). Similarly, no effect of senior therapistprior experience with the treatment protocolwas observed on measures of clinician-ratedCGI (slope: p ¼ .94, icept: p ¼ .96) or CSR(slope: p ¼ .86, icept: p ¼ .95), self-reportedSTAI (slope: p ¼ .84, icept: p ¼ .53), ortreatment discontinuation (OR ¼ 1.02, 95%CI ¼ 0.91–1.14, p ¼ .72).Junior therapist effects. As with the examin-ation of the effect of senior therapistexperience, junior therapist experienceshowed no significant relationship with out-comes as measured the clinician-rated CGI(slope: p ¼ .91, icept: p ¼ .94) or CSR (slope:p ¼ .94, icept: p ¼ .95), self-reported STAI(slope: p ¼ .56, icept: p ¼ .73), or treatmentdiscontinuation (OR ¼ 1.21, 95% CI ¼ 0.94–

VOL 43, NO 3, 2014 Trainee Experience and Outcomes 235

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

1.33, p ¼ .13). Similarly, no effect of juniortherapist prior experience with the treatmentprotocol was observed on measures ofclinician-rated CGI (slope: p ¼ .93, icept:p ¼ .90) or CSR (slope: p ¼ .94, icept:p ¼ .92), self-reported STAI (slope: p ¼ .63,icept: p ¼ .25), or treatment discontinuation(OR ¼ 1.00, 95% CI ¼ 0.83–1.20, p ¼ .98).

Discussion

This study sought to examine the impact ofpsychotherapy trainee experience, defined asboth (a) global therapist experience and (b)treatment protocol-specific experience, ontreatment outcomes following a manualized12-week transdiagnostic CBT protocol foranxiety disorders. Overall, despite therapistsranging from novice to highly experienced, nosignificant impact of therapist experience wasobserved on any indices of treatment out-comes or client dropout. These findings heldconsistent whether therapist experience wasdefined by total amount of graduate levelexperience providing psychotherapy or by thetotal amount of previous experience deliveringthe specific treatment protocol, as well aswhether experience was combined across co-therapists or examined individually by senioror junior therapist. Similarly, the lack of asignificant impact of experience on outcomesheld consistent whether examining clinician-rated outcomes from blind independentassessors or client self-reported outcomes.Our results are consistent with some of the

previous empirical literature suggesting thattrainee experience is not a significant predictorof therapy outcomes (Forand et al., 2011;Nyman et al., 2010; Schmidt et al., 2012; vanOppen et al., 2010), particularly when employ-ing manual-based cognitive-behavioral treat-ments. In contrast, however, some of thelimited literature on CBT outcomes doessuggest some impact of experience on out-comes, although the relative magnitude of theimpact is small (Driscoll et al., 2003; Huppertet al., 2001). For example, Driscoll et al. (2003)reported that therapist experience (defined asthe amount of hours and months of experiencedelivering any evidence-based treatment,regardless of specific diagnoses treated) onlyaccounted for 4% of the variability inimprovement after controlling for initialseverity. Similarly, Huppert et al. (2001)

found mixed results when assessing the impactof therapist experience on outcomes using asample of 14 therapists treating individualswith panic disorder in a large randomizedclinical trial. Although no significant effect ofoverall psychotherapy experience was observedon the primary outcome measures, two of eightsecondary outcome measures showed modestassociations with overall years of psychother-apy experience. Interestingly, years of CBTexperience were not significantly related tochange on any outcome measures (Huppertet al., 2001).Several conclusions can be offered based on

the results of the current study in the context ofthe limited additional literature on the impactof therapist experience. First, although the datafrom each of these studies were drawn fromclinical trials showing positive treatment effects,treatment outcomes and therapist experiencewere either unrelated (current study; Forandet al., 2011; Nyman et al., 2010; Schmidt et al.,2012; van Oppen et al., 2010) or modestlyrelated (Driscoll et al., 2003; Huppert et al.,2001). These non-significant or modest effectswere found whether therapist experience wasdefined as global amount of general psy-chotherapy experience, global amount ofexperience providing evidence-based psy-chotherapy, or specific amount of experiencedelivering a specific treatment protocol. Simi-larly, these modest to non-significant effectsof therapist experience were observed whenexamining psychotherapy trainees or estab-lished independent psychotherapy providers,or when examining whether outcomes werebased on a range of client diagnoses and/orpsychotherapy orientations or particular treat-ment protocols for specified treatment samples.Returning to the question posed in the

title—Does experience matter?—the answerbased on the current results and the priorliterature appears to be no, or at most verylittle, among graduate-level trainees who havecompleted at least some basic coursework inpsychopathology and clinical interventionsand were paired with a co-therapist. However,this conclusion may be somewhat superficial,as therapist experience may be seen as a proxyfor therapist quality based on the notion thatmore experienced therapists should be “bet-ter” therapists. The extent to which this is trueremains unclear, although anecdotal clinicalobservation commonly suggests that both

236 Norton, Little and Wetterneck COGNITIVE BEHAVIOUR THERAPY

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

effective and ineffective therapists existacross the full spectrum of prior psychother-apy experience. Luborsky, McLellan, Diguer,Woody, and Seligman (1997) reported that,among three therapists providing psychother-apy in multiple clinical trials, therapists whoshowed positive client outcomes in one trialshowed similarly positive outcomes in othertrials. As such, in future studies it may provevaluable to estimate therapist quality not interms of prior experience, but rather in termsof prior outcomes.

Future studies should also consider specifi-cally recruiting and randomly assigning thera-pists of a range of experience levels (e.g., earlytrainee through licensed doctoral-level thera-pists), as well from a range of clinicalpsychology (terminal M.A., PsyD, Ph.D.)and other (Master of Social Work, LicensedMarital and Family Therapist) trainingenvironments, in order to better establishcausal relationships between training/experi-ence and clinical outcomes. Further, futurestudies should endeavor to examine suchrelationships for the treatment of othercommon psychological disorders such asdepression and substance use.

Limitations should be considered whenweighing the results of this study. First, thecurrent study utilized only doctoral studenttrainee therapists, which limits the potentialrange of psychotherapy experience levels.Similarly, all therapists contributing to thetreatment trials were under supervision by thefirst author and utilizing the same treatmentprotocol, which may have limited the potentialimpact of experience levels on outcomes.Furthermore, measuring therapist experiencein years of clinical training may be a less precisemethod than using the number of direct clienthours, which has been suggested by previousresearch to be more precise (Driscoll et al.,2003). However, assessing years of therapisttraining/experiencemaybeamoregeneralizablemeasure of therapist experience, encompassinga greater variety of types of experiences (e.g.,practicum settings; clinical supervision). Inaddition, the current study also measuredtherapist experience specifically by the numberofprevious transdiagnosticCBTgroups that co-lead, which is a precise measure of experience inimplementing this specific treatment protocol.Finally, all group sessions were conducted byco-therapists, so it is possible that any

experience effects might have been mitigatedby this combination, although no effects wereobserved when “junior” and “senior” thera-pists’ experience levelswereanalyzedseparately.Even so, much of the previous literature ontherapist experience effects have utilized thera-pists with a broad range of experience levels, aswell as a variety of treatment approaches, andhave found similar results.

Limitations aside, the results of the currentstudy suggest that trainee therapists’ experi-ence, whether examined as global amount oftherapy experience or specific experiencedelivering a specific treatment protocol, wasunrelated to treatment outcomes or treatmentdiscontinuation across a range of outcomes.These results generally converge with thelimited existing literature indicating that priorexperience in providing psychotherapy ser-vices does not meaningfully influence clientoutcomes.

Acknowledgements

This research was supported by an NIMHMentored Research Scientist DevelopmentAward (MH073920) and University of Hous-ton Grant to Enhance and Advance Researchaward.

Notes* Email: telittle@uh.edu

** Email: cwetterneck@rogershospital.org

1. One outcome measure showed a significant differ-ence, although this was in favor of the master-leveltherapists.

ReferencesAmerican Psychiatric Association (2013). Diagnos-

tic and statistical manual of mental disorders (5thed.). Arlington, VA: Author.

Antony, M.M., Orsillo, S.M., & Roemer, L. (2001).Practitioner’s guide to empirically basedmeasures of anxiety. New York, NY: Plenum.

Brown, T.A., Di Nardo, P.A., & Barlow, D.H.(1994). Anxiety disorders interview schedule forDSM-IV (adult version). Albany, NY: Gray-wind.

Brown, T.A., Di Nardo, P.A., Lehman, C.L., &Campbell, L.A. (2001). Reliability of DSM-IVanxiety and mood disorders: Implicationsfor the classification of emotional disorders.Journal of Abnormal Psychology, 110, 49–58.doi:10.1037/0021-843X.110.1.49

VOL 43, NO 3, 2014 Trainee Experience and Outcomes 237

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4

Crits-Christoph, P., Connolly, M.B., Gallop, R.,Barber, J. P., Tu, X., Gladis, M., . . . &Siqueland, L. (2001). Early improvement duringmanual-guided cognitive and dynamic psy-chotherapies predicts 16-week remission status.Journal of Psychotherapy Practice and Research,10, 145–154.

Driscoll, K.A., Cukrowicz, K.C., Reitzel, L.R.,Hernandez, A., Petty, S.C., & Joiner, T.E.(2003). The effect of trainee experience inpsychotherapy on client treatment outcome.Behavior Therapy, 34, 165–177. doi:10.1016/S0005-7894(03)80011-4

Fisher, P.L., & Durham, R.C. (1999). Recoveryrates in generalized anxiety disorder followingpsychological therapy: An analysis of clinicallysignificant change in the STAI-T across out-come studies since 1990. Psychological Medi-cine, 29, 1425–1434. doi:10.1017/S0033291799001336

Forand, N.R., Evans, S., Haglin, D., & Fishman, B.(2011). Cognitive behavioral therapy in prac-tice: Treatment delivered by trainees at anoutpatient clinic is clinically effective. BehaviorTherapy, 42, 612–623. doi:10.1016/j.beth.2011.02.001

Howard, R.C. (1999). Treatment for anxietydisorders: Does specialty training help?Professional Psychology: Research and Practice,30, 470–473. doi:10.1037/0735-7028.30.5.470

Huppert, J.D., Bufka, L.F., Barlow, D.H., Gor-man, J.M., Shear, M.K., &Woods, S.W. (2001).Therapists, therapist variables, and cognitive-behavioral therapy outcome in a multicentertrial for panic disorder. Journal of Consultingand Clinical Psychology, 69, 747–755. doi:10.1037//0022-006X.69.5.747

Little, R.J., &Rubin, D.B. (2002). Statistical analysiswith missing data (2nd ed.). New York, NY:Wiley.

Luborsky, L., McLellan, A.T., Diguer, L., Woody,G., & Seligman, D.A. (1997). The psychothera-pist matters: Comparison of outcomes acrosstwenty-two therapists and seven patientsamples. Clinical Psychology, 4, 53–65. doi:10.1111/j.1468-2850.1997.tb00099.x

Muthen, L.K., & Muthen, B.O. (2006). Mplususers’guide. Los Angeles, CA: Muthen &Muthen.

National Institute of Mental Health (1985). Clinicalglobal impressions scale. PsychopharmacologyBulletin, 21, 839–843.

Norton, P.J. (2008). An open trial of a transdiag-nostic cognitive-behavioral group therapy foranxiety disorder. Behavior Therapy, 39,242–250. doi:10.1016/j.beth.2007.08.002

Norton, P.J. (2012a). A randomized clinical trial oftransdiagnostic CBT for anxiety disorder bycomparison to relaxation training. BehaviorTherapy, 43, 506–517. doi:10.1016/j.beth.2010.08.011

Norton, P.J. (2012b). Group cognitive-behavioraltherapy of anxiety: A transdiagnostic treatmentmanual. New York, NY: Guilford.

Norton, P.J., & Barrera, T.L. (2012). Transdiag-nostic versus diagnosis-specific CBT for anxietydisorders: A preliminary randomized controlledtrial. Depression and Anxiety, 29, 874–882.doi:10.1002/da.21974

Norton, P.J., & Hope, D.A. (2005). Preliminaryevaluation of a broad-spectrum cognitive-behavioral group therapy for anxiety. Journalof Behavior Therapy and Experimental Psychia-try, 36, 79–97. doi:10.1016/j.jbtep.2004.07.002

Norton, P.J., & Price, E.C. (2007). A meta-analyticreview of adult cognitive-behavioral treatmentoutcome across the anxiety disorders. TheJournal of Nervous and Mental Disease, 195,521–531. doi:10.1097/01.nmd.0000253843.70149.9a

Nyman, S.J., Nafziger, M.A., & Smith, T.B. (2010).Client outcomes across counselor training levelwithin a multitiered supervision model. Journalof Counseling and Development, 88, 204–209.doi:10.1002/j.1556-6678.2010.tb00010.x

Powell, J.L., Hunter, H.L., Beasley, L.O., &Vernberg, E.M. (2010). Using fine-grainedindexes of therapists’ experience and trainingto predict treatment outcomes in a university-based training clinic for children and families.Training and Education in Professional Psychol-ogy, 4, 138–144. doi:10.1037/a0018369

Schmidt, N.B., Buckner, J.D., Pusser, A., Wool-away-Bickel, K., & Preston, J.L. (2012).Randomized controlled trial of False SafetyBehavior Elimination Therapy (F-SET): Aunified cognitive behavioral treatment foranxiety psychopathology. Behavior Therapy,43, 518–532.

Spielberger, C.D. (1983).Manual for the State-TraitAnxiety Inventory (STAI). PaloAlto, CA:Consulting Psychologists Press.

Spielberger, C.D., Gorsuch, R.L., Luschene, R.E.,Vagg, P.R., & Jacobs, G.A. (1993). State-TraitAnxiety Inventory for adults. Palo Alto, CA:Mind Garden.

Swift, J.K., & Greenberg, R.P. (2012). Prematurediscontinuation in adult psychotherapy:A meta-analysis. Journal of Consulting andClinical Psychology, 80, 547–559. doi:10.1037/a0028226

van Oppen, P., van Balkom, A.J.L.M., Smit, J.H.,Schuurmans, J., van Dyck, R., & Emmelkamp,P.M.G. (2010). Does the therapy manual or thetherapist matter most in treatment of obsessive-compulsive disorder? A randomized controlledtrial of exposure with response or ritualprevention in 118 patients. Journal of ClinicalPsychiatry, 71, 1158–1167. doi:10.4088/JCP.08m04990blu

238 Norton, Little and Wetterneck COGNITIVE BEHAVIOUR THERAPY

Dow

nloa

ded

by [

Mem

oria

l Uni

vers

ity o

f N

ewfo

undl

and]

at 0

9:52

14

July

201

4