Embed Size (px)
Citation preview
19th Annual RTC ConferencePresented in Tampa, February 2006
1
Validity of Treatment TargetProgress Ratings as Indicators of
Youth Improvement
Brad Nakamura & Charles MuellerUniversity of Hawaii at Manoa
Introduction• Practitioners are increasingly required to demonstrate and
document intervention outcomes• Individualized measurement strategies
– Target complaints (TC; Battle et al., 1966)– Goal attainment scaling (GAS; Kiresuk et al., 1994)
• Monthly Treatment & Progress Summary Form(MTPS; Child and Adolescent Mental Health Division;2003)– Service format– Service setting– Treatment targets– Clinical progress ratings– Intervention practice elements– Provider outcomes
Battle, Imber, Hoehn-Sario, Nash, & Frank, 1966; Kiresuk, Smith, & Cardillo, 1994
Introduction
Other:ShynessPositive PeerInteractionHyperactivityCommunity
Involvement
Other:Sexual MisconductPositive FamilyFunctioningHealth ManagementCognitive-Intellectual
Functioning
Willful Misconduct,Delinquency
Self-InjuriousBehaviorPersonal HygieneGriefAvoidance
TreatmentEngagementSelf-ControlPeer/ Sibling ConflictGender Identity
ProblemsAttention Problems
Traumatic StressSchool Refusal/TruancyPeer InvolvementFire SettingAssertiveness
SuicidalitySchool InvolvementOppositional/ Non-Compliant BehaviorEnuresis, EncopresisAnxiety
Substance UseRunawayMedical RegimenAdherenceEmpathyAnger
Speech andLanguage ProblemsPsychosisManiaEating, Feeding
ProblemsAggression
Social SkillsPositive Thinking/AttitudeLow Self-EsteemDepressed MoodAcademic
Achievement
Sleep DisturbancePhobias/FearsLearning Disorder,Underachievement
Contentment,Enjoyment,Happiness
Activity Involvement
Treatment Targets
Introduction
Clinical Progress Ratings
PositiveFamilyFunctioning
DepressedMood
OppositionalBehavior
CompleteImprovement
91-100%
SignificantImprovement
71-90%
ModerateImprovement
51-70%
SomeImprovement
31-50%
MinimalImprovement
11-30%
NoSignificantChanges0-10%
Deterioration< 0%
Introduction
• Purpose of the present study– Examine and compare the degree of change
over the course of treatment as assessed bytwo measures• Monthly Treatment and Progress Summary form
(MTPS; Child and Adolescent Mental HealthDivision, 2003)
• Child and Adolescent Functional AssessmentScale (CAFAS; Hodges, 1998)
Introduction
Hypotheses1. There should be no meaningful relationship
(i.e., lack of a significant correlation)between MTPS and CAFAS scores at intakeinto the CAMHD system
2. There should be a significant inverserelationship between MTPS and CAFASscores at six-month follow-up (i.e., afterreceiving six months of services)
19th Annual RTC ConferencePresented in Tampa, February 2006
2
MethodParticipants (N = 74)• Age
– M = 13.82– SD = 2.99– Range = 4.40 to 18.19
• Gender– 42 males (56.8%)– 32 females (43.2%)
• Principal diagnoses– Mood/anxiety (37.8%)– Disruptive behavior (24.3%)– Attentional (23.0%)
Method
Measures• Child and Adolescent Functional
Assessment Scale (CAFAS; Hodges,1998)
• Monthly Treatment and ProgressSummary (MTPS; Child and AdolescentMental Health Division; 2003)– Treatment targets– Clinical progress ratings
Method
Procedure• Participants were selected on the basis of the
availability of both CAFAS and MTPS reports attwo separate times
1. Intake into our system2. At six-month follow-up
Follow-up MTPS reportIntake MTPS reportMTPSFollow-up CAFAS reportIntake CAFAS reportCAFAS
Six-month follow upIntake into system
MethodProcedure• CAFAS scores
– Hodges’ (1998) eight-scale scoring procedure• MTPS scores
– Mean progress rating scores were derived byaveraging progress rating scores for all stabletargets
Follow-up MTPS scoreIntake MTPS scoreMTPSFollow-up CAFAS scoreIntake CAFAS scoreCAFAS
Six-month follow upIntake into system
Results
AcademicAchievement (24%)
AcademicAchievement (35%)
AcademicAchievement (35%)
Depressed Mood(26%)
Depressed Mood(36%)
OppositionalBehavior (45%)
OppositionalBehavior (34%)Anger (47%)Anger (46%)
Anger (36%)OppositionalBehavior (55%)
TreatmentEngagement 50%)
Positive FamilyFunctioning (53%)
Positive FamilyFunctioning (74%)
Positive FamilyFunctioning (55%)
MostCommonTargets (n)
4.15 (2.16)7.23 (2.34)6.39 (2.51)Target: M (SD)
Stable6-MonthIntake
Results
AcademicAchievement (24%)
AcademicAchievement (35%)
AcademicAchievement (35%)
Depressed Mood(26%)
Depressed Mood(36%)
OppositionalBehavior (45%)
OppositionalBehavior (34%)Anger (47%)Anger (46%)
Anger (36%)OppositionalBehavior (55%)
TreatmentEngagement 50%)
Positive FamilyFunctioning (53%)
Positive FamilyFunctioning (74%)
Positive FamilyFunctioning (55%)
MostCommonTargets (n)
4.15 (2.16)7.23 (2.34)6.39 (2.51)Target: M (SD)
Stable6-MonthIntake
19th Annual RTC ConferencePresented in Tampa, February 2006
3
Results
AcademicAchievement (24%)
AcademicAchievement (35%)
AcademicAchievement (35%)
Depressed Mood(26%)
Depressed Mood(36%)
OppositionalBehavior (45%)
OppositionalBehavior (34%)Anger (47%)Anger (46%)
Anger (36%)OppositionalBehavior (55%)
TreatmentEngagement 50%)
Positive FamilyFunctioning (53%)
Positive FamilyFunctioning (74%)
Positive FamilyFunctioning (55%)
MostCommonTargets (n)
4.15 (2.16)7.23 (2.34)6.39 (2.51)Target: M (SD)
Stable6-MonthIntake
Results
1.623.081.222.02MTPS
36.688.030.9109.9CAFAS
SDMSDM
6-Month Follow UpIntake
t (73) = -5.06, p < .001, Effect Size = 0.71
Note. CAFAS = Child and Adolescent Functional Assessment Scale score,MTPS = Monthly Treatment Provider Summary score.
Results
1.623.081.222.02MTPS
36.688.030.9109.9CAFAS
SDMSDM
6-Month Follow UpIntake
t (73) = 4.77, p < .001, Effect Size = 0.87
Note. CAFAS = Child and Adolescent Functional Assessment Scale score,MTPS = Monthly Treatment Provider Summary score.
Results
CAFAS intakescore
CAFAS 6-monthfollow-up score
MTPS 6-monthfollow-up score
MTPS intakescore
.01
*p < .05. **p < .01.
Results
*p < .05. **p < .01.
CAFAS intakescore
CAFAS 6-monthfollow-up score
MTPS 6-monthfollow-up score
MTPS intakescore
.12
Results
*p < .05. **p < .01.
CAFAS intakescore
CAFAS 6-monthfollow-up score
MTPS 6-monthfollow-up score
MTPS intakescore
.10
-.06
19th Annual RTC ConferencePresented in Tampa, February 2006
4
Results
*p < .05. **p < .01.
CAFAS intakescore
CAFAS 6-monthfollow-up score
MTPS 6-monthfollow-up score
MTPS intakescore
.40**
Results
*p < .05. **p < .01.
CAFAS intakescore
CAFAS 6-monthfollow-up score
MTPS 6-monthfollow-up score
MTPS intakescore
-.28*
Results
*p < .05. **p < .01.
CAFAS intakescore
CAFAS 6-monthfollow-up score
MTPS 6-monthfollow-up score
MTPS intakescore
Partial correlation for 6-month follow-up CAFASand MTPS scores, holdingconstant CAFAS and MTPSintake scores
[-.34**]
DiscussionLimitations• MTPS/CAFAS analyses limited to six-month relationship• Small sample• Further study is needed for examining factors that
influence data completionNext Steps• Examine the relationship between MTPS scores and
scores from other standardized measures of symptoms(e.g., Child Behavior Checklist; Achenbach & Rescorla,2001)
• Examine treatment target- or diagnoses-specificrelationships
Conclusion• Nomothetically calculated change scores for
youths’ improvements on idiographic treatmenttargets (i.e., MTPS scores) may serve as validmeasures of client change
• The magnitude of these relationships (r = -.34)suggests some level of specificity for both theCAFAS and MTPS measures
• Multi-level systems utility– Reasoning at individual case-load level– Practitioner accountability– System-wide trends
Thank You!
Mahalo!
19th Annual RTC ConferencePresented in Tampa, February 2006
5
Results
-.07.09379 months
-.19.06486 months
-.36**.031043 months
CALOCUS
-.43**.14489 months
-.34**.01746 months
-.29**.111283 months
CAFAS
Hypothesis 2:Partial Correlation for Standard
Measure and MTPS Follow-Up, holding intake scores
constant
Hypothesis 1:Standard Measure
Intake to MTPS IntakenIntake to
BenchmarkMeasure
*p < .05. **p < .01.Note. CAFAS = Child and Adolescent Functional Assessment Scale, CALOCUS = Child and Adolescent Level of Care Utilization System
